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Defense  Weather  Satellites:  Analysis  of  Alternatives  Is  Useful  for  Certain  Capabilities,  but 
Ineffective  Coordination  Limited  Assessment  of  Two  Critical  Capabilities 

The  Department  of  Defense  (DOD)  uses  data  from  military,  U.S.  civil  government,  and 
international  partner  satellite  sensors  to  provide  critical  weather  information  and  forecasts  for 
military  operations.  As  DOD’s  primary  existing  weather  satellite  system — the  Defense 
Meteorological  Satellite  Program  (DMSP) — ages  and  other  satellites  near  their  estimated  end  of 
life,  DOD  faces  potential  gaps  in  its  space-based  environmental  monitoring  (SBEM)  capabilities 
which  may  affect  stakeholders  that  use  SBEM  data,  including  the  military  services,  the 
intelligence  community,  and  U.S.  civil  agencies  such  as  the  National  Oceanic  and  Atmospheric 
Administration  (NOAA).  After  two  unsuccessful  attempts  to  develop  follow-on  programs  from 
1997  through  fiscal  year  201 2,  including  the  National  Polar-orbiting  Operational  Satellite  System 
(NPOESS),  a  tri-agency  program  between  DOD,  NOAA,  and  the  National  Aeronautics  and 
Space  Administration  that  was  canceled  in  2010  because  of  extensive  cost  overruns  and 
schedule  delays,  DOD  and  other  stakeholders  who  rely  on  SBEM  data  are  now  in  a  precarious 
position  in  which  key  capabilities  require  immediate  and  near-term  solutions.1  With  potential 
capability  gaps  starting  as  early  as  this  year,  it  is  important  for  DOD  to  make  decisions  in  a 
timely  manner,  but  based  on  informed  analysis  that  considers  stakeholder  input. 

From  February  2012  through  September  2014,  DOD  conducted  a  requirements  review  and  an 
analysis  of  alternatives  (AOA)  to  identify  and  compare  the  operational  effectiveness  and  life 
cycle  costs  of  potential  solutions  for  providing  SBEM  capabilities.2  An  AOA — a  key  analysis  in 
DOD’s  acquisition  process — is  intended  to  inform  a  decision  on  the  most  cost  effective  solution 
for  meeting  validated  capability  requirements  and  identify  a  wide  range  of  solutions  with  a 
reasonable  likelihood  of  providing  the  needed  capabilities.  DOD  conducted  the  SBEM  AOA  in 
two  phases: 

•  Phase  1,  which  took  place  from  October  2012  into  March  2013,  examined  12  potential 
capability  gaps  to  determine  the  military  utility — or  operational  benefit — and  operational 
risk  of  each  gap. 


May  1994,  a  Presidential  Decision  Directive  required  DOD  and  the  Department  of  Commerce  through  NOAA  to 
converge  their  two  separate  weather  satellite  programs  into  a  single  program  capable  of  satisfying  both  military  and 
civilian  requirements.  Presidential  Decision  Directive  NSTC-2,  Convergence  of  U.S.  Polar-Orbiting  Operational 
Environmental  Satellite  Systems  (May  5,  1994).  The  NPOESS  program  started  in  1997;  after  it  was  cancelled  in  2010 
DOD  was  given  responsibility  for  covering  the  early  morning  polar  orbit  and  started  a  separate  program,  the  Defense 
Weather  Satellite  System.  The  program  was  cancelled  in  fiscal  year  2012  because  of  cost  and  timing  considerations. 
NOAA  has  developed  the  Joint  Polar  Satellite  System  to  meet  its  responsibility  for  coverage  in  the  afternoon  orbit. 

2  This  period  includes  the  development  of  study  guidance  and  a  study  plan  for  the  AOA,  and  the  Joint  Requirement 
Oversight  Council’s  review  of  the  AOA.  The  analysis  took  approximately  10  months  to  complete,  from  October  2012 
into  July  2013,  followed  by  3  months  to  prepare  the  final  AOA  report,  issued  in  October  2013. 
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•  Phase  2,  which  took  place  from  March  2013  into  July  2013,  assessed  potential  solutions 
for  meeting  the  gaps,  focusing  on  3  of  the  capabilities  identified  in  Phase  1  as  potentially 
requiring  a  space-based  solution. 

The  Carl  Levin  and  Howard  P.  “Buck”  McKeon  National  Defense  Authorization  Act  (NDAA)  for 
Fiscal  Year  2015  included  a  provision  for  us  to  review  the  SBEM  AOA.3  This  report  formally 
transmits  the  briefing  we  provided  to  congressional  staff  on  October  20  and  21 , 2015,  as  well  as 
the  final  results  of  our  work.  See  enclosure  I:  Department  of  Defense  (DOD)  Weather  Satellites: 
Briefing  to  Congressional  Defense  Committees.  The  briefing  addressed  (1)  The  extent  to  which 
the  SBEM  AOA  addressed  input  from  stakeholders  and  assessed  the  range  of  alternatives  for 
potential  solutions  and  (2)  the  extent  to  which  the  AOA  informed  DOD’s  plans  for  providing 
SBEM  capabilities. 

In  addition,  Senate  Report  No.  114-49  to  accompany  S.  1376,  a  bill  for  the  NDAA  for  Fiscal 
Year  2016,  included  a  provision  for  us  to  evaluate  whether  launching  the  last  DMSP  satellite, 
DMSP-20,  is  the  most  cost-effective  solution  for  covering  an  expected  SBEM  data  gap  over  the 
Indian  Ocean;  however,  in  conducting  this  review,  we  found  that  there  was  limited  cost 
information  to  make  this  determination.  Moreover,  during  the  course  of  our  audit  work,  DOD 
moved  to  terminate  DMSP-20  in  December  2015  due  to  lack  of  funding  for  fiscal  year  201 6. 4 
Nevertheless,  because  the  launch  of  DMSP-20  is  related  to  issues  raised  by  the  AOA,  the 
enclosed  briefing  slides  discuss  potential  impacts  of  not  launching  the  satellite.  Also,  the 
relatively  high  costs  of  integrating,  storing,  and  launching  DMSP-20 — estimated  at  over  $400 
million — have  been  important  factors  in  weighing  whether  to  launch  the  last  satellite.  Storage 
costs  alone,  including  the  costs  of  maintaining  contractor  support  and  a  launch-readiness 
posture,  have  accounted  for  a  significant  portion  of  these  costs — our  related  recent  report  found 
the  costs  of  storing  DMSP-20  to  be  $40  million  annually.5 

To  determine  the  extent  to  which  the  SBEM  AOA  addressed  the  input  of  stakeholders  and 
assessed  the  range  of  alternatives  for  potential  solutions,  we  reviewed  relevant  DOD  and  GAO 
documents  to  develop  an  understanding  of  the  requirements  and  guidance  for  conducting  an 
AOA.  We  reviewed  the  AOA  documents  and  interviewed  DOD  officials  involved  in  conducting 
and  reviewing  the  AOA  to  understand  how  it  was  developed.  We  also  interviewed  users  and 
providers  of  DOD  SBEM  data  (stakeholders),  such  as  military  service,  intelligence  community, 
and  NOAA  officials,  to  gain  their  perspectives  on  how  stakeholder  views  were  incorporated  into 
the  AOA.  We  also  interviewed  industry  officials  about  ways  to  effectively  assess  options  for 
providing  SBEM  capabilities.  To  determine  the  extent  to  which  the  AOA  informed  plans  for 
providing  SBEM  capabilities,  we  reviewed  documents  and  interviewed  DOD  officials  about  plans 
and  decision  making  processes  for  providing  future  SBEM  capabilities,  including  decisions 
about  launching  DMSP-20,  the  coverage  it  may  provide,  and  related  cost  estimates,  to  the 
extent  they  were  available.  We  also  reviewed  documents  and  interviewed  NOAA  officials  about 
activities  of  the  international  SBEM  community,  including  international  partners,  to  understand 
those  potential  effects  on  DOD’s  plans. 


3Pub.  L.  No.  113-291  §1612(a)(3)  (2014). 

4The  Explanatory  Statement  of  the  Consolidated  Appropriations  Act,  2016,  Pub.  L.  No.  114-113  (2015)  provided  fora 
$89,351,000  reduction  of  the  DMSP  fiscal  year  2016  budget  request,  and  a  rescission  of  $50,000,000  from  the  fiscal 
year  2015  procurement  funds  for  DMSP.  161  Cong.  Rec.  H9693  at  H9928  (Dec.  17,2015). 

5GAO,  Space  Acquisitions:  DOD  Needs  More  Guidance  on  Decisions  to  Store  Satellites,  GAO-1 5-97R  (Washington, 
D.C.:  Dec.  9,  2014). 
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We  conducted  this  performance  audit  from  May  2015  to  March  2016  in  accordance  with 
generally  accepted  government  auditing  standards.  Those  standards  require  that  we  plan  and 
perform  the  audit  to  obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that  the  evidence  obtained 
provides  a  reasonable  basis  for  our  findings  and  conclusions  based  on  our  audit  objectives. 

In  summary,  DOD  conducted  a  thorough  assessment  of  some  capabilities,  but  pressures  to 
complete  the  AOA  in  time  to  inform  budget  planning  and  decision  making  for  near-term  needs 
imposed  limitations  on  the  analysis  of  the  military  utility  of  the  two  highest-priority  capabilities  in 
Phase  1  of  the  AOA.  In  addition,  DOD  did  not  effectively  collaborate  with  NOAA — the  agency 
that,  on  a  case-by-case  basis,  represents  DOD’s  interests  in  international  partnerships 
regarding  SBEM  data.  The  lack  of  formal  coordination  and  collaboration  with  NOAA,  such  as 
employing  a  mechanism  that  identified  roles  and  responsibilities  for  the  two  agencies  during  the 
AOA,  contributed  to  an  incorrect  assumption  about  the  continued  availability  of  critical  data  from 
European  satellites.  As  a  result,  cloud  characterization  and  theater  weather  imagery  were  not  as 
thoroughly  analyzed  for  potential  solutions  in  Phase  2.  Despite  the  limitations  of  the  AOA,  it 
offered  analysis  that  was  useful  for  informing  plans  for  a  space-based  solution  for  three 
capabilities  with  near-term  needs.  In  addition,  the  AOA  determined  that  most  of  the  remaining 
eight  capabilities  with  military  utility  could  be  covered  by  other  assets  or  addressed  with 
modeling  development.6  However,  the  AOA  is  less  useful  as  a  decision  making  tool  in  support 
of  plans  for  the  two  highest-priority  capabilities,  cloud  characterization  and  theater  weather 
imagery  data,  for  which  it  did  not  fully  evaluate  potential  solutions. 

The  SBEM  AOA  Was  Thorough  in  Some  Areas,  but  Limited  Stakeholder  Coordination 
Affected  the  Range  of  Alternatives  Assessed 

Ideally,  DOD  could  have  conducted  an  AOA  for  SBEM  capabilities  when  it  was  pursuing  new 
acquisitions  in  the  aftermath  of  the  NPOESS  cancellation  in  2010,  in  order  to  provide  analysis 
with  a  long-term  focus.  Instead,  DOD  initiated  an  AOA  in  2012  with  the  goal  of  having  the 
results  of  the  analysis  in  time  to  plan  for  the  fiscal  year  2015  budget.  There  is  no  standard  time 
frame  for  conducting  an  AOA,  and  DOD  officials  have  stated  that  AOAs  are  always  limited  in 
time  and  resources.  However,  in  prior  work,  we  have  concluded  that  conducting  AOAs  under 
compressed  time  frames  in  order  to  meet  a  planned  milestone  decision  or  weapon  system 
fielding  date  can  short-change  a  comprehensive  assessment  of  risks.7  Further,  we  concluded 
such  AOAs  may  be  completed  too  late  in  the  process  to  inform  effective  trade  discussions 
before  a  program  begins  development.  Despite  time  constraints,  DOD  generally  conducted  a 
thorough  review  of  the  military  utility  of  capabilities  in  Phase  1 ,  including  assessing  the 
effectiveness  of  DOD  providing,  or  not  providing,  a  space-based  solution  for  each  capability. 
However,  pressures  to  complete  the  AOA  in  time  to  inform  budget  planning  and  decision 
making  for  near-term  needs  imposed  limitations  on  the  analysis  of  the  two  highest-priority 
capabilities — cloud  characterization  and  theater  weather  imagery — during  Phase  1  of  the  AOA. 
This,  in  turn,  affected  the  extent  to  which  these  capabilities  were  analyzed  further  for  solutions 
during  Phase  2.  For  example,  according  to  officials,  the  risk  assessment  of  these  two 


6The  AOA  study  team  determined  that  1  capability — auroral  characterization — out  of  the  12  capabilities  initially 
identified,  was  not  being  used  operationally  and  so  did  not  warrant  further  assessment,  leaving  1 1  capabilities  to 
consider  for  Phase  2. 

7ln  a  review  of  over  20  programs  with  AOAs,  we  found  that  7  of  the  programs  conducted  AOAs  that  were  limited 
because  of  compressed  time  frames  or  concurrency  with  other  planning  activities.  GAO,  Defense  Acquisitions:  Many 
Analyses  of  Alternatives  Have  Not  Provided  a  Robust  Assessment  of  Weapon  System  Options,  GAO-09-665 
(Washington,  D.C.:  Sept.  24,  2009). 
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capabilities  used  a  limited  weather  data  set  because  of  the  time  required  to  generate  and 
process  data. 

The  AOA  team  effectively  engaged  with  a  broad  range  of  DOD  stakeholders,  though  concerns 
were  raised  about  the  methodology  used  to  assess  some  gaps  in  both  phases  of  the  AOA. 
Further,  ineffective  coordination  with  NOAA — which  generally  is  the  signatory  to  international 
agreements  on  space-based  weather  monitoring  data  sharing  and  represents  DOD’s  interests 
with  international  partners  on  a  case-by-case  basis — affected  the  range  of  capabilities  that  were 
fully  assessed  in  Phase  2  of  the  AOA.8  Specifically,  limited  engagement  with  NOAA  contributed 
to  an  incorrect  assumption  about  the  continued  availability  of  critical  data  from  European 
satellites.  The  Air  Force’s  AOA  Handbook  advises  AOA  leaders  to  identify  appropriate 
stakeholders  and  how  they  will  be  involved  in  the  study  and  review  process,  including  vetting 
results  of  the  final  analysis.9  However,  DOD  did  not  have  a  formal  coordination  and 
collaboration  mechanism  that  specified  roles  and  responsibilities  for  DOD  and  NOAA  in 
conducting  the  AOA.  NOAA  was  not  involved  in  reviews  of  the  AOA  or  regular  discussions  with 
AOA  study  leadership,  according  to  NOAA  officials,  and  did  not  provide  formal  input  during  the 
AOA  study  period.  While  DOD  made  some  efforts  to  communicate  with  NOAA  representatives 
about  European  coverage  over  the  Indian  Ocean — the  particular  region  of  concern — such  as 
through  a  discussion  with  a  senior  technical  advisor,  these  efforts  did  not  result  in  DOD 
determining  the  full  risks  of  assuming  that  European  coverage  would  continue.  Moreover,  during 
the  time  of  the  AOA  study,  publicly  available  reports  from  an  international  coordination  group 
indicated  uncertainty  about  extended  European  coverage  over  the  Indian  Ocean.10  Despite  this 
information,  and  the  study  team’s  inability  to  quantify  the  risk  of  relying  on  international  partners 
in  the  future,  the  final  AOA  report  determined  that  the  likelihood  the  gap  would  not  be  filled  was 
low,  based  on  historical  trends.  As  a  result,  the  Office  of  the  Secretary  of  Defense,  Cost 
Analysis  and  Program  Evaluation,  and  the  Undersecretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  determined  that  the  AOA  team  did  not  need  to  fully  assess  space- 
based  solutions  for  the  two  highest-priority  capabilities  because  it  assumed  that  support  from 
civil  and  international  partners  would  suffice. 

The  analysis  in  Phase  1  of  the  AOA  informed  the  range  potential  solutions,  or  alternatives,  to  be 
assessed  in  Phase  2.  Based  on  the  Phase  1  results,  DOD  chose  to  fully  assess  alternatives  for 
3  capabilities  with  near-term  needs.  For  the  remaining  8  capabilities  with  military  utility,  DOD 
determined  they  either  had  limited  space-based  utility  or  would  be  sufficiently  provided  by 
climatology  and/or  ground-based  or  U.S.  civil  government  and  international  partner  capabilities, 
and  therefore  would  undergo  an  abbreviated  Phase  2  assessment.  Within  the  scope  of  three 
capabilities  to  be  fully  assessed,  the  range  of  options  DOD  evaluated  was  broad  and  included 
alternatives  beyond  replacing  the  existing  system  with  the  same  concept.  In  prior  work,  based 
on  an  assessment  of  over  20  programs  that  conducted  AOAs,  we  found  that  programs  that 
employed  such  a  practice  tended  to  result  in  better  cost  and  schedule  outcomes  than  programs 
that  examined  a  limited  range  of  options.11  DOD  assessed  alternatives  for  the  remaining  eight 


8For  example,  NOAA  represents  DOD  interest  on  programs  such  as  the  Joint  Polar  System. 

9Office  of  Aerospace  Studies,  Air  Force  Materiel  Command  OAS/A9,  Analysis  of  Alternatives  (AOA)  Handbook:  A 
Practical  Guide  to  Analyses  of  Alternatives  (Kirtland  Air  Force  Base,  N.Mex.:  July  2010). 

i  n 

Coordination  Group  for  Meteorological  Satellites,  Report  of  the  41st  Meeting  of  the  Coordination  Group  for 
Meteorological  Satellites  (Tsukuba,  Japan:  July  8-12,  2013);  EUMETSAT’s  Plans  for  Indian  Ocean  Coverage  Beyond 
2013,  CGMS-41  EUM-WP-15  via  (July  2,  2013);  and  Report  of  the  40th  Meeting  of  the  Coordination  Group  for 
Meteorological  Satellites  (Lugano,  Switzerland:  Nov.  5-8,  2012). 

11  GAO-09-665. 
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capabilities  to  a  limited  extent,  but  did  not  assess  alternatives  for  providing  cloud 
characterization  and  theater  weather  imagery — deemed  the  highest-priority  capabilities — as  fully 
as  the  three  capabilities  with  near-term  needs. 

The  AOA  Informed  Plans  for  Some  Capabilities,  but  Is  Less  Useful  as  a  Decision  Making 
Tool  for  Two  Critical  Capabilities 

DOD  made  an  effort  to  plan  for  future  capabilities  with  a  more  cost-effective  approach  in  mind, 
including  considering  which  capabilities  DOD  needed  to  provide  and  which  could  be  provided  by 
leveraging  other  sources  of  data.  For  the  3  capabilities  with  near-term  needs,  the  AOA  offered 
thorough  analysis  that  was  useful  for  informing  related  plans.  Based  on  the  analysis,  DOD  is 
developing  plans  for  the  Weather  System  Follow-on  (WSF)  to  provide  ocean  surface  vector 
wind  and  tropical  cyclone  intensity  capabilities,  though  it  may  not  be  available  in  time  to  avoid 
short-term  gaps.  For  example,  the  current  and  only  system  that  fully  meets  DOD’s  needs  for 
ocean  surface  vector  wind  data  is  expected  to  reach  end  of  life  as  early  as  this  year.  However, 
the  planned  WSF  technology  demonstration,  which  DOD  officials  said  will  partially  meet  the 
requirements,  is  estimated  to  launch  in  2017,  and  the  WSF  system  intended  to  fully  meet 
requirements  is  estimated  to  launch  in  2022.  For  the  third  capability,  energetic  charged 
particles,  DOD  has  developed  a  plan  to  collect  data  by  hosting  sensors  on  other  satellites. 

The  AOA  is  less  useful  for  informing  plans  for  the  two  highest-priority  capabilities  for  which  it  did 
not  fully  assess  solutions —  cloud  characterization  and  theater  weather  imagery  data,  now 
facing  near-term  gaps  over  the  Indian  Ocean.12  Outside  of  the  AOA  process,  DOD  is  now 
examining  short-term  options  for  mitigating  those  gaps,  which  include: 

•  continuing  to  rely  on  DMSP  coverage  through  the  end  of  life  for  DMSP-19,13  and 

•  obtaining  data  from  European  and  Indian  partners.14 

Longer-term  plans  for  providing  these  capabilities  are  undetermined,  though  DOD  is  exploring 
options,  which  include  continued  or  increased  reliance  on  data  provided  by  international 
partners.  Given  NOAA’s  role  in  coordinating  with  and  obtaining  space-based  environmental 
monitoring  data  from  international  partners,  DOD  and  NOAA  officials  have  indicated  that  formal 
collaboration  is  key  to  ensuring  DOD’s  interests  are  represented  in  international  partnerships. 
Since  the  conclusion  of  the  AOA  study  period  in  the  fall  of  2013,  DOD  and  NOAA  have 
increased  their  communication  by  discussing  ways  to  leverage  international  partners’  satellite 
data  and  the  possibility  of  establishing  and  employing  formal  coordination  and  collaboration 
arrangements.  One  potential  vehicle  for  formalizing  collaboration  is  the  recently  re-established 
Committee  for  Operational  Environmental  Satellites,  led  by  NOAA’s  Office  of  the  Federal 
Coordination  for  Meteorology.  While  the  committee  does  not  necessarily  focus  on  international 


12The  Air  Force  has  been  directed  by  the  Joint  Requirements  Oversight  Council  to  include  performance  parameters 
and  mitigation  strategies,  including  solutions  that  do  not  require  the  development  or  purchase  of  new  DOD  items,  for 
cloud  characterization  and  theater  weather  imagery  in  the  capability  development  document  for  the  DOD  solution. 
According  to  some  officials,  related  efforts  include  the  review  and  re-assessment  of  assumptions  in  the  AOA. 

In  addition,  DOD  is  still  responsible  for  providing  weather  monitoring  coverage  in  the  early  morning  polar  orbit.  With 
the  termination  of  DMSP-20,  that  coverage  is  currently  estimated  to  last  until  2020  and  DOD  has  less  time  to 
determine  how  it  will  continue  to  meet  that  need. 

1 4 

In  July  2015,  DOD  also  inquired  with  NOAA  about  the  possibility  of  using  one  of  NOAA’s  geostationary  weather 
satellites  to  preserve  coverage  over  the  Indian  Ocean.  NOAA  responded  that,  while  technically  feasible,  potential 
limitations  would  need  to  be  addressed,  including  ensuring  the  availability  of  a  satellite  as  a  spare. 
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partner  issues,  DOD  officials  have  stated  that  it  is  one  way  DOD  can  connect  with  NOAA’s 
international  affairs  officials.  Our  body  of  work  on  interagency  collaboration  has  shown  that 
establishing  formal  collaborative  mechanisms  and  documenting  agreements  can  help  provide 
clarity  about  roles  and  responsibilities  and  support  accountability.15 

Conclusions 

DOD’s  effort  to  analyze  options  in  the  SBEM  AOA,  including  consideration  of  ways  to  leverage 
other  sources  of  data,  is  a  positive  step  toward  a  more  cost-effective  approach  to  providing 
SBEM  capabilities,  and  for  some  of  these  capabilities,  the  SBEM  AOA  is  useful  for  decision 
making.  However,  understanding  the  limitations  of  the  SBEM  AOA  is  important  for  informing 
sound  decision  making.  As  a  result  of  the  AOA’s  limitations,  as  well  as  cancellations  of  prior 
efforts  to  develop  a  follow-on  system  to  DMSP,  DOD  is  in  a  position  in  which  it  has  to  quickly 
initiate  efforts  to  assess  potential  solutions  for  other  near-term  capability  gaps  that  were  not  fully 
assessed  in  the  AOA.  Because  decisions  about  whether  to  provide  DOD  solutions  for  SBEM 
capabilities  are  dependent  on  the  availability  of  data  from  U.S.  civil  government  and 
international  partner  satellites,  sufficient  and  reliable  information  to  determine  the  level  of  risk 
DOD  is  willing  to  take  is  crucial.  Formalizing  coordination  and  collaboration  to  identify  roles  and 
responsibilities  in  planning  for  SBEM  capabilities  could  offer  DOD  and  NOAA  the  opportunity  to 
help  ensure  effective  communication  about  the  availability  and  reliability  of  data  from  U.S.  civil 
government  and  international  partner  satellites  and  better  inform  decision  making  in  the  future. 

Recommendation  for  Executive  Action 

To  help  ensure  DOD  is  sufficiently  informed  about  the  availability  and  reliability  of  data  from 
U.S.  civil  government  and  international  partner  satellites  as  it  plans  for  future  SBEM  capabilities 
that  rely  on  such  satellites,  we  recommend  that  the  Secretary  of  Defense  ensure  the  leads  of 
future  SBEM  planning  efforts  establish  formal  mechanisms  for  coordination  and  collaboration 
with  NOAA  that  specify  roles  and  responsibilities  and  ensure  accountability  for  both  agencies. 

Agency  Comments  and  Our  Evaluation 

DOD  and  NOAA,  through  the  Department  of  Commerce,  provided  us  written  comments  on  a 
draft  of  this  report.  These  comments  are  reprinted  in  enclosures  II  and  III.  DOD  and  NOAA  also 
provided  technical  comments  that  have  been  incorporated  where  appropriate. 

In  its  written  comments,  DOD  disagreed  with  statements  in  our  draft  report  that  detailed 
alternatives  were  only  developed  for  three  capabilities — ocean  surface  vector  wind,  tropical 
cyclone  intensity,  and  energetic  charged  particles.  Specifically,  DOD  stated  that  the  AOA 
developed  alternatives  capable  of  meeting  all  12  capability  gaps  and  examined  the  military  utility 
assessed  with  the  operational  risk  of  all  12  gaps.  We  acknowledge  that  the  alternatives  included 
all  capabilities  assessed  in  the  SBEM  AOA  that  were  determined  to  have  military  utility. 

However,  the  level  of  analysis  applied  to  alternatives  for  the  three  capabilities  with  nearest-term 
needs  was  more  detailed  and  thorough  than  the  analysis  of  alternatives  for  the  remaining 
capabilities,  including  the  two  highest-priority  capabilities:  cloud  characterization  and  theater 
weather  imagery.  The  AOA  report  itself  acknowledged  the  focus  of  analysis  on  these  three 
capabilities.  DOD  analyzed  a  total  of  10  alternatives — 1  representing  the  baseline  option  in 


15GAO,  Managing  for  Results:  Key  Considerations  for  Implementing  Interagency  Collaborative  Mechanisms , 
GAO-12-1022  (Washington,  D.C.:  Sept.  27,  2012);  and  Interagency  Collaboration:  Implications  of  a  Common 
Alignment  of  World  Regions  among  Select  Federal  Agencies,  GAO-1 1-776R  (Washington,  D.C.:  July  1 1 , 201 1 ). 


Page  6 


GAO-1 6-252R  Weather  Satellite  Alternatives 


which  DOD  would  rely  only  on  non-DOD  assets  to  provide  all  capabilities;  7  primarily  focused  on 
various  ways  to  provide  ocean  surface  vector  wind,  tropical  cyclone  intensity,  and  energetic 
charged  particles;  and  2  that  sought  to  address  all  capabilities  either  by  leveraging  alternative 
data  sources  as  a  more  cost-effective  approach  to  mitigating  gaps,  or  by  providing  a  DOD 
solution  that  would  meet  all  minimum  performance  requirements.16  While  decision  makers  could 
have  decided,  on  their  own  accord,  to  isolate  the  potential  solutions  for  cloud  characterization 
and  theater  weather  imagery  from  the  latter  two  alternatives,  the  range  of  options  explored  for 
those  two  capabilities  were  minimal  as  compared  with  the  range  of  options  for  the  three 
capabilities  that  were  fully  assessed.  Specifically,  in  each  of  the  two  alternatives  that  included 
cloud  characterization  and  theater  weather  imagery,  one  option  was  assessed  for  providing 
those  capabilities.  By  comparison,  seven  alternatives  were  focused  on  various  ways  to  provide 
ocean  surface  vector  wind,  tropical  cyclone  intensity,  and  energetic  charged  particles.  The  AOA 
laid  out  a  valuable  rationale  for  focusing  on  the  capabilities  that  DOD  really  needed  to  provide 
instead  of  moving  forward  with  DOD  providing  all  capabilities  on  its  own.  Accordingly,  we 
revised  statements  as  appropriate  so  as  not  to  imply  that  all  12  capabilities  were  not  assessed 
early  on  in  the  analysis  or  that  all  12  warranted  a  DOD  solution.  However,  it  is  clear  DOD 
missed  an  opportunity  to  more  fully  assess  options  for  cloud  characterization  and  theater 
weather  imagery.  As  a  result,  DOD  is  now  re-assessing  ways  to  meet  the  near-term  needs  of  its 
two  highest-priority  SBEM  capabilities. 

DOD  also  disagreed  with  statements  in  our  draft  report  that  time  constraints  imposed  limitations 
on  the  analysis  and  its  ability  to  inform  decisions  about  whether  to  provide  DOD  solutions.  DOD 
stated  that  the  Senior  Advisory  Group  that  guided  the  AOA  proceeded  methodically  and  moved 
forward  to  new  stages  in  the  study  only  after  achieving  unanimous  agreement  that  the 
preceding  stages  had  been  completed.  In  addition,  DOD  stated  that  the  Study  Advisory  Group 
granted  additional  time  when  the  Navy  raised  concerns  about  one  requirement’s  minimum 
acceptable  value.  We  recognize  that  AOAs  are  generally  limited  by  time  and  resource 
constraints,  and  that  DOD  was  faced  with  both  constraints  in  conducting  the  SBEM  AOA.  Our 
statements  regarding  time  constraints  are  to  acknowledge  this  reality  and  to  make  transparent 
the  fact  that  time  limitations  had  an  effect  on  the  extent  to  which  the  AOA  team  could  conduct 
some  pieces  of  the  analysis,  such  as  exploring  the  utility  of  certain  capabilities  in  additional 
planning  scenarios  and  vignettes.  While  it  is  not  possible  to  determine  the  potential  outcomes 
such  additional  analysis  may  have  generated,  it  is  important  to  reiterate  that  in  prior  work  we 
have  concluded  that  AOAs  conducted  under  compressed  time  frames  have  been  limited,  and 
may  not  have  enough  time  to  assess  a  broad  range  of  alternatives  and  their  risks.17  As  such,  we 
believe  limitations  related  to  time  constraints  should  be  acknowledged.  However,  to  avoid 
implying  that  time  constraints  were  the  root  cause  of  all  the  issues  we  raise  in  our  findings,  we 
revised  our  report  title  and  made  other  edits  as  appropriate,  including  noting  that  the  Navy  was 
provided  additional  time  to  conduct  analysis. 

In  addition,  DOD  disagreed  with  statements  in  our  draft  report  regarding  modeling  limitations. 
Specifically,  DOD  stated  that  the  AOA  used  meteorological  models  that  are  currently  used  with 


16Ocean  surface  vector  wind,  tropical  cyclone  intensity,  and  energetic  charged  particles  were  assessed  under  one 
principal  alternative  with  six  excursions,  fora  total  of  seven  alternatives. 

17GAO-09-665. 


Page  7 


GAO-1 6-252R  Weather  Satellite  Alternatives 


the  DOD  Weather  Centrals  and  by  civil  meteorological  partners.18  We  do  not  contend  that  the 
AOA  did  not  use  current  meteorological  models;  rather,  and  in  keeping  with  the  AOA  findings, 
we  acknowledge  that  some  forecast  models  were  determined  to  be  insufficient  to  demonstrate 
the  advantages  of  using  space-based  measurements  to  close  gaps  that  were  determined  to 
have  military  utility.  Multiple  officials  we  interviewed  confirmed  this  finding  and  stated  that  further 
development  of  models  could  help  better  leverage  data  sources  going  forward.  However,  we 
recognize  that  some  capabilities,  such  as  snow  depth  and  soil  moisture,  have  inherent 
limitations  in  being  measured  from  space,  and  that  the  time  required  to  develop  sufficient 
models  is  difficult  to  predict.  We  believe  that  one  of  the  positive  outcomes  of  the  AOA  was  to 
identify  more  cost-effective  steps  DOD  could  take  to  mitigate  potential  capability  gaps,  such  as 
developing  algorithms  to  more  effectively  process  existing  data.  Accordingly,  we  revised  our 
report  so  as  not  to  suggest  that  a  longer  period  for  the  AOA  study  could  have  enabled  the 
necessary  modeling  development. 

DOD  also  disagreed  with  statements  in  our  draft  report  that  the  AOA  did  not  fully  capture  the 
risk  that  coverage  by  international  partner  satellite  systems,  such  as  Europe’s  Meteosat,  might 
not  continue.  DOD  stated  that  the  AOA  was  fully  informed  of  the  status  of  future  plans  of  all  civil 
and  international  partner  systems,  and  that  the  AOA  team  had  standing  members  from  NOAA  to 
ensure  the  team’s  analyses  were  informed  by  current  data  on  those  systems.  We  continue  to 
uphold  our  finding  that  DOD  did  not  effectively  coordinate  with  NOAA  officials  to  help  ensure  the 
department  had  sufficient  information  about  plans  for  Meteosat  and  to  adequately  assess  the 
related  risk.  While  several  NOAA  officials  were  assigned  to  one  of  the  AOA  working  groups, 
according  to  one  NOAA  participant,  the  interaction  entailed  receiving  emails  rather  than 
participating  in  meetings  or  regular  dialogue  throughout  the  AOA.  Regarding  the  status  of  and 
plans  for  international  partner  systems  in  particular,  NOAA  officials  said  they  provided  copies  of 
their  memorandums  of  agreement  with  international  partners  to  DOD,  but  DOD  did  not  make 
further  attempts  to  discuss  the  issue  with  NOAA’s  international  affairs  office,  which  regularly 
interacts  with  international  partners.  DOD  did  consult  with  a  knowledgeable  representative 
serving  as  a  senior  technical  advisor  to  NOAA  at  the  time  and  discussed  potential  future 
coverage  over  the  Indian  Ocean,  among  other  issues.  However,  the  results  of  this  discussion 
were  presented  to  the  AOA  team  as  it  was  approaching  Phase  2  of  the  study,  at  which  point  the 
assumption  about  continued  support  from  international  partners  had  already  been  established 
and  used  for  the  analysis,  and  the  team  was  moving  forward  with  assessing  alternatives 
primarily  focused  on  other  capabilities.19  Further,  the  representative  said  he  told  DOD  that 
although  the  European  agency  had  supported  coverage  over  the  Indian  Ocean  region  in  the 
past,  there  was  no  obligation  to  provide  that  coverage  again  in  the  future.  During  the  AOA  study 
period,  publicly  available  reports  confirmed  this  status,  indicating  uncertainty  about  extended 
Meteosat  coverage  of  the  region  and  stating  that  the  European  agency  had  not 


18The  DOD  Weather  Centrals  include  the  557th  Weather  Wing  (formerly  the  Air  Force  Weather  Agency),  the  Fleet 
Numerical  Meteorology  and  Oceanography  Center,  the  Naval  Oceanographic  Office,  and  NOAA’s  National 
Environmental,  Satellite,  Data,  and  Information  Service.  The  centrals  ingest  and  exploit  weather  data  from  satellites 
on  a  daily  basis. 

1 Q 

The  final  AOA  report  states,  as  the  first  driving  assumption  for  the  study,  that,  “Baseline  civil/international  SBEM 
satellites  will  provide  capabilities,  as  designed,  for  the  duration  of  their  projected  lifetimes  and  this  data  will  be 
available  during  the  planning  period.  This  assumption  also  implies  that  Europe,  Japan,  and  the  United  States  will 
have  follow-on  systems  that  match  current  capabilities,  with  the  exception  that  there  will  be  no  follow-on  to  DMSP 
after  the  end  of  life  of  DMSP-20.”  It  further  states,  “This  means  there  will  always  be  geostationary  coverage”  of 
several  regions,  including  the  Indian  Ocean,  “by  U.S.  and  European  satellites.” 
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committed  to  providing  coverage  in  the  long  term.20  While  the  AOA  study  guidance  directed  the 
study  team  to  conduct  a  comprehensive  assessment  of  existing  and  planned  future  international 
partner  capabilities  and  the  likelihood  of  their  availability,  the  AOA  team  ultimately  did  not 
assess  future  plans  and,  as  directed  by  the  Study  Advisory  Group,  did  not  attempt  to  quantify 
the  risk.  Despite  this  limitation  and  the  information  available  at  the  time,  the  AOA  team  deemed 
the  likelihood  of  the  Indian  Ocean  gap  not  being  filled  as  low,  based  on  historical  trends. 

In  concurring  with  our  recommendation,  DOD  stated  that  the  recommendation  is  consistent  with 
current  working-level  engagements  between  DOD  and  NOAA,  and  that  it  will  invite  senior  NOAA 
representatives  to  participate  in  future  SBEM  planning  efforts  and  maintain  frequent  contact  with 
NOAA  during  SBEM  acquisition  efforts  to  ensure  DOD  is  informed  of  the  plans  and  operational 
status  of  international  partner  systems.  This  is  a  positive  step.  However,  we  believe  DOD  should 
formalize  its  coordination  and  collaboration  in  future  efforts,  whether  through  committees  that 
have  been  established  or  re-established  since  the  AOA  study  was  completed,  such  as  the 
Committee  for  Operational  Environmental  Satellites,  or  other  means,  to  include  clearly  defined 
roles  and  responsibilities  for  the  two  agencies. 

In  its  written  comments,  NOAA  stated  that  our  draft  report  “captures  the  key  issues  relating  to 
the  usefulness  of  the  AOA  including  calling  into  question  the  understanding  of  risk  that  was 
accepted,”  and  that  our  draft  report  “noted  that  NOAA  leadership  was  not  involved  in  the 
reviews  of  the  AOA  or  any  regular  advisory  group  meetings.” 


We  are  sending  copies  of  this  report  to  the  appropriate  congressional  committees,  the  Secretary 
of  Defense,  and  the  Director  of  NOAA.  In  addition,  the  report  is  available  at  no  charge  on  the 
GAO  website  at  http://www.gao.gov. 

Should  you  or  your  staff  have  any  questions  concerning  this  report,  contact  me  at  (202)  512- 
4841  or  at  chaplainc@gao.gov.  Contact  points  for  our  Offices  of  Congressional  Relations  and 
Public  Affairs  may  be  found  on  the  last  page  of  this  report.  Key  contributors  to  this  report  are 
Rich  Horiuchi,  Assistant  Director;  Jay  Tallon,  Assistant  Director;  Maricela  Cherveny;  Erin 
Cohen;  Brenna  Guarneros;  Krista  Mantsch;  Marie  Ahearn,  Emily  Bond,  and  Roxanna  Sun. 


Cristina  T.  Chaplain 

Director,  Acquisition  and  Sourcing  Management 
Enclosures  -  3 


on 

These  reports  were  published  by  the  Coordination  Group  for  Meteorological  Satellites,  which  has  a  range  of 
international  member  organizations,  including  NOAA  and  the  European  agency  responsible  for  the  Meteosat  system, 
EUMETSAT.  Coordination  Group  for  Meteorological  Satellites,  Report  of  the  41st  Meeting  of  the  Coordination  Group 
for  Meteorological  Satellites'  EUMETSAT’s  Plans  for  Indian  Ocean  Coverage  Beyond  201 3;  and  Report  of  the  40th 
Meeting  of  the  Coordination  Group  for  Meteorological  Satellites. 
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Introduction 

•  Meteorological  and  oceanographic  data — also  referred  to  as 
weather  data — are  key  to  providing  information  for  the 
successful  planning,  execution,  and  sustainment  of  U.S. 
military  operations.  Some  weather  data  are  collected  through 
space-based  environmental  monitoring  (SBEM)  satellites.  The 
Department  of  Defense  (DOD)  uses  data  from  its  own 
satellites,  such  as  the  Defense  Meteorological  Satellite 
Program  (DMSP),  as  well  as  from  U.S.  civil  government  and 
international  partner  satellites  to  provide  weather  information 
and  forecasting. 
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Introduction  (continued) 

•  As  existing  systems  age,  DOD  is  planning  for  follow-on 
systems  to  avoid  gaps  in  critical  capabilities.  From  February 
2012  to  September  2014,  DOD  conducted  a  requirements 
review  and  analysis  of  alternatives  (AOA)  for  SBEM-related 
capabilities  to  identify  potential  solutions.1  The  AOA  and 
subsequent  decisions  could  affect  stakeholders  that  use 
SBEM  data,  including  the  military  services,  the  intelligence 
community,  and  U.S.  civil  agencies  such  as  the  National 
Oceanic  and  Atmospheric  Administration  (NOAA).  With 
potential  capability  gaps  starting  as  early  as  this  year,  it  is 
important  for  DOD  to  make  decisions  n  a  timely  manner,  but 
based  on  informed  analysis  that  considers  stakeholders’  input. 


analysis  of  alternatives  (AOA)  is  a  key  analysis  in  the  DOD  acquisition  process  that  compares  the  operational  effectiveness,  suitability,  and  lifecycle  costs  of  solutions  to  satisfy 
documented  capability  needs.  This  period  includes  the  development  of  study  guidance  and  a  study  plan  for  the  AOA,  and  the  Joint  Requirement  Oversight  Council’s  review  of  the  AOA. 
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Introduction  (continued) 


•  The  National  Defense  Authorization  Act  (NDAA)  for  Fiscal  Year 
2015  included  a  provision  for  GAO  to  review  the  SBEM  AOA.2 

•  Senate  Report  No.  114-49  to  accompany  S.  1376,  a  bill  for  the 
NDAA  for  Fiscal  Year  2016,  included  a  provision  for  GAO  to 
evaluate  whether  launching  the  final  DMSP  satellite,  DMSP-20,  was 
the  most  cost-effective  solution  for  covering  an  expected  SBEM 
data  gap  over  the  Indian  Ocean.  At  the  time,  questions  had  been 
raised  about  whether  to  launch  the  satellite  in  light  of  concerns 
about  its  cost  and  utility  given  its  age  and  military  needs.  In  October 
2015  GAO  provided  information  in  oral  briefings  to  congressional 
committees  regarding  DOD’s  plans  for  DMSP-20.  Since  then,  DOD 
has  indicated  it  will  not  launch  DMSP-20  due  to  lack  of  funding. 


2Carl  Levin  and  Howard  P.  “Buck”  McKeon  National  Defense  Authorization  Act  for  Fiscal  Year  2015,  Pub.  L.  No.  113-291  §161 2(a)(3)  (2014). 
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Objectives 

This  briefing  addresses  the  following  questions: 

1 )  To  what  extent  did  the  SBEM  AOA  address  input  from 
stakeholders  and  assess  the  range  of  alternatives  for 
potential  solutions? 

2)  To  what  extent  has  the  AOA  informed  DOD’s  plans  for 
providing  SBEM  capabilities? 
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Summary 


GAO  found: 

1 .  The  AOA  team  conducted  a  thorough  assessment  of  some  capabilities, 
despite  pressures  to  complete  the  AOA  in  time  to  inform  decision  making  for 
near-term  needs.  However,  time  constraints  imposed  limitations  on  the 
analysis  of  the  military  utility  of  certain  critical  capabilities,  which  affected  the 
extent  to  which  they  were  assessed  in  the  range  of  alternatives.  The  AOA 
team  also  engaged  with  a  broad  range  of  DOD  stakeholders,  though  some 
stakeholder  concerns  remained  unaddressed  in  the  AOA.  In  addition,  DOD 
did  not  effectively  coordinate  with  NOAA — the  agency  that,  on  a  case  by  case 
basis,  represents  DOD’s  interests  in  international  partnerships  regarding 
SBEM  data.  This  was  partly  due  to  a  lack  of  a  formal  coordination  and 
collaboration  mechanism  that  identified  roles  and  responsibilities  for  DOD 
and  NOAA  and  facilitated  input  and  discussion  during  the  AOA.  For  example, 
NOAA  was  not  involved  in  reviews  of  the  AOA  or  regular  discussions  with 
study  leadership,  according  to  officials.  DOD  made  an  incorrect  assumption 
about  continued  availability  of  international  support,  which  we  believe  could 
have  been  avoided  if  coordination  was  more  effective.  Based  on  the  incorrect 
assumption,  DOD  decided  not  to  assess  the  two  highest-priority  capabilities 
more  fully  for  potential  DOD  solutions  in  the  range  of  alternatives. 
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Summary  (continued) 


GAO  found: 

2)  The  AOA  informed  plans  for  providing  a  space-based  solution  for  three 
capabilities  with  near-term  needs.  To  address  these  needs,  the  Air  Force  is 
currently  developing  plans  for  a  follow-on  system  and  for  hosting  sensors  on  Air 
Force  satellites.  For  most  of  the  remaining  nine  capabilities  assessed,  the  AOA 
determined  that  other  assets  could  provide  coverage,  or  that  modeling 
development  could  help  mitigate  potential  gaps.  Flowever,  the  AOA  is  less  useful 
as  a  decision  making  tool  for  the  two  highest-priority  capabilities  for  which  it  did 
not  fully  evaluate  potential  solutions.  Outside  of  the  AOA  process,  DOD  is  now 
examining  options  for  meeting  these  two  needs  in  the  short  term,  including 
increased  reliance  on  data  from  international  partner  satellites.  Because  DOD  is 
still  assessing  those  options,  data  on  the  most  cost-effective  approach  are  not  yet 
available.  Longer-term  plans  for  providing  the  two  highest-priority  capabilities  are 
undetermined.  Given  NOAA’s  role  in  coordinating  with  international  partners  to 
obtain  SBEM  data,  and  limited  mechanisms  for  formal  coordination  and 
collaboration  between  DOD  and  NOAAthat  clarify  roles  and  responsibilities  and 
provide  for  mutual  accountability,  further  assessments  of  options  that  rely  on  data 
from  international  partner  satellites  may  be  at  risk  of  again  using  incomplete  or 
incorrect  information. 
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Background:  Satellites  Providing  SBEM 
Coverage 

•  Satellites  carrying  sensors  that  collect  SBEM  data  primarily  include: 

•  Polar-orbiting  satellites  in  low  Earth  orbit  constantly  circle  the  earth  in  an  almost 
north-south  orbit  over  the  poles.  Each  successive  orbital  pass  occurs  at  the  same 
local  time  of  day,  such  as  early  morning,  mid-morning,  and  afternoon.3  These 
satellites  provide  global  high-resolution  observations  ideal  for  tactical  weather 
support  and  long  range  numerical  weather  prediction. 

•  Geostationary  satellites  maintain  a  fixed  position  relative  to  the  earth,  collecting 

data  on  a  specific  geographic  region  and  providing  real-time  monitoring  of 
developing  weather  conditions  (see  fig.  1).  c.  .....  . ._  .  ,0  ,  ....  . 

•  SBEM  sensors  collect  data  on  a 
range  of  weather-related  factors, 
such  as  cloud  cover,  winds, 
precipitation,  atmospheric 
temperature,  sea  ice  conditions, 
and  space  weather  that  may  affect 
the  accuracy  and  operability  of 
other  space-based  signals. 


Polar  and  Geostationary  Orbits 


i:  GAO  depiction  of  notional  polar  and  geostationary  orbits  (data);  Map  Resources  (map).  |  GAO-16-252R 


3ln  addition  to  satellites  in  polar  orbit,  satellites  in  other  low  Earth  orbits  provide  SBEM  data  to  operational  users. 
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Background:  Satellites  Providing  SBEM 
Coverage  (continued) 

•  Since  the  1960s,  the  United  States  has  operated  two  separate 
meteorological  polar-orbiting  satellite  systems: 

•  DOD’s  DMSP — Currently  crossing  the  equator  in  the  early 
and  mid-morning  orbits. 

•  NOAA’s  Polar-orbiting  Operational  Environmental  Satellite 
(POES)  and  Suomi  National  Polar-orbiting  Partnership 
satellite,  the  first  in  the  Joint  Polar  Satellite  System — 
Currently  crossing  the  equator  in  the  afternoon  orbit. 

•  The  United  States  also  relies  on  a  European  satellite,  the 
Meteorological  Operational  satellite,  currently  crossing  the 
equator  in  the  mid-morning  orbit. 
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Background:  Satellites  Providing  SBEM 
Coverage  (continued) 

•  Currently  the  DOD  relies  on  SBEM  data  from  geostationary  satellites  primarily  provided  by 
U.S.  civil  government,  Europe,  and  Japan,  each  sharing  data  from  a  specific  geographic  region 
in  order  to  provide  global  coverage  (see  fig.  2). 4  Such  satellites  provide  high  levels  of  effective 
coverage  between  40  and  50  degrees  north  or  south,  depending  on  geographic  location. 


Figure  2:  Geostationary  Satellites  Providing  SBEM  Coverage  (Notional) 


Source:  GAO  depiction  of  notional  coverage  of  geostationary  weather  satellites;  Map  Resources  (map).  |  GAO-16-252R 

4ln  addition,  DOD  accesses  data  from  a  meteorological  payload  on  a  South  Korean  geostationary  satellite.  Indian,  Chinese,  and  Russian  geostationary  satellites  also  collect 
weather  data,  but  DOD  does  not  currently  use  these  sources. 
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Background:  History  of  DMSP  Replacement 
Efforts 

•  DOD  has  been  involved  in  two  previous  efforts  to  develop  a 
replacement  for  DMSP,  both  of  which  were  cancelled: 

•  National  Polar-orbiting  Operational  Environmental  Satellite 
System  (NPOESS) — Tri-agency  program  between  DOD,  NOAA, 
and  the  National  Aeronautics  and  Space  Administration  to 
replace  both  DMSP  and  POES;  started  in  1997  and  cancelled  in 
2010  due  to  escalating  costs  and  schedule  delays.5  (see  fig.  3) 

•  Defense  Weather  Satellite  System — DOD  program  intended  to 
continue  providing  weather  observations  from  the  morning  orbit 
following  NPOESS  cancellation;  started  in  2010  and  cancelled  in 
fiscal  year  2012  because  the  program  was  considered  early-to- 
need  with  unsustainable  costs. 

5DOD,  through  the  Air  Force,  was  responsible  for  the  NPOESS  acquisition,  NOAA  was  responsible  for  overall  program  management  and  satellite  operations,  and  the  National 
Aeronautics  and  Space  Administration  was  responsible  for  facilitating  the  development  and  incorporation  of  new  technologies. 
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Background:  History  of  DMSP  Replacement 

Efforts  (continued) 

Figure  3:  Time  Line  of  Efforts  to  Replace  DMSP 


1962  - ih 

Defense  Meteorological 
Satellite  Program 
(DMSP)  ongoing 


1960  - 

Polar  Orbiting 
Environmental 
Satellites  (POES) 
ongoing 


i  k 1994 

DOD  and  NOAA  required  to 
develop  new  combined  system, 
the  National  Polar-orbiting 
Operational  Environmental 
Satellite  System  (NPOESS) 


1  Department  of  Defense  (DOD) 

— i  National  Oceanic  and  Atmospheric  Administration  (NOAA) 
Source:  GAO  analysis  of  DOD  and  NOAA  information.  |  GAO-16-252R 


2010 

Defense  Weather  Satellite 
System  (DWSS)  program 
start 


2012  (fiscal  year) 
DWSS  cancelled  due 
to  costs  and  timing 
considerations 


► 


2002 

Primary  NPOESS 
contract  awarded 


2010 

NPOESS  cancelled  due 
to  escalating  costs  and 
schedule  delays 


2010 

NOAA/NASA  Joint 
Polar  Satellite  System 
Program  (JPSS)  start 


2011 

JPSS  program 
launches  Suomi 
National  Polar-orbiting 
Partnership  satellite 


► 


Page  14 

GAO-1 6-252R  DOD  Weather  Satellite  Alternatives 


Page  24 


GAO 


Background:  SBEM  Requirements  Review  and 
AOA 


Following  the  cancellation  of  the  Defense  Weather  Satellite 
System,  DOD  conducted  follow-on  program  activities. 

•  In  2012,  the  Air  Force  analyzed  almost  250  weather  data 


lection  parameters,  or  characteristics,  from  a  2009 
ia  capabilities  document  and  determined. that  about 


co 
■  ■ 

ini 

100  parameters  required  space-based  sensing. 

The  Air  Force  then  organized  those  parameters  into 
three  categories  based  on  priority  and  ultimately 
identifiedl 2  potential  gaps  in  capability  that  were  mission 
critical,  insufficiently  met  by  existing  and  planned 
sources,  and  required  space-basea  sensing  capability. 
Officials  representing  the  military  services  and  the 
intelligence  community  jointly  prioritized  these  12 
capabi  ities,  which  subsequently  formed  the  list  of 
capabi  ities  that  were  to  be  assessed  in  the  AOA. 
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Background:  SBEM  Requirements  Review  and 

AOA  (continued) 


•  An  AOA  is  a  key  analysis  intended  to  support  DOD’s  acquisition  process. 
Specifically,  an  AOA  is  intended  to  inform  a  decision  on  the  most  cost 
effective  solution  for  meeting  validated  capability  requirements,  and  is  to 
identify  a  wide  range  of  solutions  with  a  reasonable  likelihood  of  providing 
the  needed  capability.  Based  on  the  preceding  requirements  review,  the  Air 
Force  conducted  an  AOA  from  2012-2013  in  two  phases.6 


•  Phase  1  examined  the  12  potential  capability  gaps 
utility — or  operational  benefit — and  operational  risk 
table  1). 


to  determine  military 
of  each  gap  (see 


•  Phase  2  assessed  potential  solutions  for  meeting  the  gaps,  focusing  on 
3  of  the  capabilities  identified  in  Phase  1  as  potentially  requiring  a 
space-based  solution. 


•  The  Joint  Requirements  Oversight  Council  reviewed  the  AOA  in  September 
2014  and  supported  the  AOA’s  findings,  including  the  determination  that  3 
of  the  12  potential  capability  gaps  could  be  considered  for  a  DOD  space- 
based  solution. 

6The  Office  of  the  Secretary  of  Defense,  Cost  Assessment  and  Program  Evaluation  (CAPE),  was  the  approval  authority  for  the  AOA  study  and  was  responsible  for  developing  study 
guidance,  approving  the  study  plan,  providing  study  oversight,  and  reviewing  the  AOA  for  sufficiency. 
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Background:  Table  1  -  Capability  Areas 
Assessed  in  the  AO  A 


Capability  Area  and  Priority  Rank 

Examples  of  Military  Mission  Areas 

Current  Partners 

DOD  Space-based  Solution  Potentially  Required?  jj 

1.  Cloud  Characterization 

Flying  operations,  mission  planning, 
long  range  strikes 

NOAA,  Japan, 
Europe 

No,  but  risk  increases  if  civil  and  international  data  sources  are 
unavailable. 

2.  Theater  Weather  Imagery 

Military  operations,  resource 
protection,  air  refueling 

NOAA,  Japan, 
Europe 

No,  but  risk  increases  if  civil  and  international  data  sources  are 
unavailable. 

3.  Ocean  Surface  Vector  Winds 

Resource  protection,  evacuation  and 
ship  maneuver  operations 

Europe 

Yes,  with  change  in  minimal  acceptable  values  for  refresh  rate  and 
timeliness.3 

4.  Ionospheric  Density 

Communications,  GPS  guided 
systems,  radar  operations 

Taiwan/NOAA 

No,  minimal  contribution  to  increased  military  utility.  Improved  models 
have  potential  to  increase  the  utility  of  data. 

5.  Snow  Depth 

Flood  estimates,  river  gap  crossing, 
ground  maneuvers 

Japan 

No,  assuming  availability  of  and  access  to  international  capability. 

Limited  contribution  from  space  due  to  measurement  uncertainty;  may 
benefit  from  investment  in  algorithm  development. 

6.  Soil  moisture 

Army  off  road  mobility,  land  operations  Japan 

No,  assuming  availability  of  and  access  to  international  capability. 

Limited  contribution  from  space  due  to  measurement  uncertainty;  may 
benefit  from  investment  in  algorithm  development. 

7.  Equatorial  Ionospheric 
Scintillation 

Communications,  GPS 

Taiwan/NOAA 

No,  space-based  solution  adds  minimal  military  utility  if  sufficient 
ground-based  sensing  is  available. 

8.  Tropical  Cyclone  Intensity 

Resource  protection,  evacuation  and 
ship  maneuver  operations 

Japan 

Yes,  with  change  in  minimal  acceptable  value  for  refresh  rate. 

9.  Sea  Ice  Characterization 

Operational  risk  and  safety  for  Arctic 
submarine  and  surface  operations 

Japan 

No,  with  operational  work-arounds. 

10.  Auroral  Characterization 

No  space-based  auroral  characterization  information  is  currently  used  operationally. 

11.  Energetic  Charged  Particle 
Characterization 

Satellite  anomaly  assessments  and 
space  protection 

Europe 

Yes,  with  change  in  minimal  acceptable  values  for  resolution,  energy, 
and  refresh  rate. 

12.  Electric  field 

Space  surveillance,  missile  defense 
radar  operations,  communication 

Taiwan/NOAA 

No,  minimal  contribution  to  increased  military  utility.  Improved  models 
have  potential  to  increase  the  utility  of  data. 

Source:  GAO  analysis  of  Department  of  Defense  information.  |  GAO-1 6-252R 

aA  refresh  rate  is  a  performance  measure  of  the  frequency  at  which  a  sensor  or  multiple  sensors  can  revisit  a  certain  area  of  coverage. 
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Objective  1:  AOA  Was  Thorough  in  Some 
Areas,  but  Limited  Stakeholder  Coordination 
Affected  the  Range  of  Alternatives  Assessed 
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Objective  1:  AOATeam  Faced  Time  Constraints 


•  The  AOA  team  faced  pressures  to  complete  the  study  in  time 
to  inform  decision  making  for  near-term  needs. 

•  Ideally,  DOD  could  have  conducted  an  AOA  for  SBEM 
capabilities  when  pursuing  new  acquisitions  after  the 
cancellation  of  NPOESS  in  2010.  Instead,  in  2012  DOD 
initiated  the  AOA  with  the  goal  of  having  the  results  in  time 
to  plan  for  the  fiscal  year  2015  budget  and  to  address  near- 
term  needs. 

•  At  start  of  the  AOA  in  201 2,  the  gap  for  ocean  surface 
vector  wind  was  expected  to  begin  as  early  as  2015. 
Other  gaps,  such  as  soil  moisture  and  tropical  cyclone 
intensity  data,  were  estimated  to  begin  as  early  as  2024 
at  the  time. 
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Objective  1:  Despite  Time  Constraints,  AOA 
Included  Thorough  Analysis 

•  The  AOA  team  generally  conducted  a  thorough  review  of  the 
military  utility  of  capabilities  in  Phase  1.  Specifically: 

•  On  a  capability-by-capabilitv  basis,  the  team  assessed  the 
effectiveness  of  DOD  providing,  or  not  providing,  a  space-based 
solution. 

•  The  AOA  team  consulted  with  warfighter  subject  matter  experts 
and  assembled  analytical  working  groups  in  key  areas,  including 
effectiveness  analysis,  technology  and  alternatives,  and 
operational  concepts.  These  working  groups: 

•  developed  specific  measures  of  performance,  effectiveness, 
and  outcomes  for  the  relevant  capabilities; 

•  developed  a  set  of  architectural  elements  that  could  meet  or 
contribute  to  needs  for  each  capability; 

•  defined  the  overarching  operating  concept  for  applying 
environmental  monitoring  to  DOD  operations;  and 

•  collaborated  with  one  another  in  conducting  their  analyses. 
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Objective  1:  Time  Constraints  Imposed 
Limitations  on  Analysis  of  Certain  Capabilities 

•  However,  pressures  to  complete  the  AOA  study  in  time  to  inform 
decision  making  limited  the  military  utility  analysis  of  the  two 
highest-priority  capabilities — cloud  characterization  and  theater 
weather  imagery — in  Phase  1  of  the  AOA.7  This  limitation  affected 
the  extent  to  which  the  capabilities  were  analyzed  further  for 
solutions  in  Phase  2.  For  example: 

•  To  assess  the  impact  of  various  satellite  and  sensor 

configurations  on  forecast  models  for  cloud  characterization  and 
theater  weather  imagery  capabilities,  the  analysis  used  a  limited 
weather  data  set,  covering  7  days,  due  to  the  time  required  to 
generate  and  process  data,  according  to  DOD  officials.  This 
affected  the  degree  to  which  operational  risk  could  be  assessed 
because  the  seasonal  variability  of  weather  conditions  is  not 
captured  in  a  1-week  data  set,  according  to  officials. 

7The  analysis  took  approximately  10  months  to  complete,  from  October  2012  into  July  2013,  followed  by  3  months  to  prepare  the  final  AOA  report,  issued  in  October  2013. 
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Objective  1:  Time  Constraints  Imposed  Limitations  on 

Analysis  of  Certain  Capabilities  (continued) 

•  Further,  DOD  officials  stated  that  in  some  cases  they  would 
have  preferred  to  test  some  capabilities  in  additional  planning 
scenarios  and  vignettes  to  further  assess  their  military  utility, 
such  as  in  additional  regions,  but  were  limited  by  time  and 
resource  constraints. 

•  There  is  a  no  standard  time  frame  for  conducting  an  AOA, 
and  DOD  officials  have  stated  that  AOAs  are  always  limited 
in  time  and  resources.  However,  in  prior  work,  we  nave 
concluded  that  conducting  AOAs  under  compressed  time 
frames  in  order  to  meet  planned  milestones  or  weapons 
system  fielding  dates  can  short-change  a  comprehensive 
assessment  of  risks  and  preclude  effective  cost,  schedule, 
and  performance  trade-offs  made  before  a  program  begins 
development.8 

8GAO,  Defense  Acquisitions:  Many  Analyses  of  Alternatives  Have  Not  Provided  a  Robust  Assessment  of  Weapon  System  Options,  GAO-09-665  (Washington,  D.C.:  Sept.  24,  2009). 
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Objective  1 :  AOA  Team  Engaged  a  Wide  Range 
of  DOD  Stakeholders;  Some  Concerns  Raised 

•  The  AOA  effort  required  engagement  with  a  wide  range  of  DOD 
stakeholders  because  of  the  broad  applicability  of  space-based 
weather  needs  across  the  services. 

•  While  the  AOA  team  effectively  engaged  with  a  broad  range  of 
DOD  stakeholders,  concerns  were  raised  about  the 
methodology  used  for  assessing  potential  gaps  in  both  AOA 
phases.  For  example: 

•  The  intelligence  community  had  concerns  related  to  how 
cloud  characterization  was  assessed.  For  example, 
representatives  stated  certain  measures  used  in  the  analysis 
did  not  effectively  demonstrate  the  utility  of  cloud 
characterization  in  Phase  1 ,  which  contributed  to  a  decision 
not  to  more  fully  assess  the  capability  in  Phase  2.  The 
intelligence  community  ultimately  agreed  to  the  final  AOA 
report  because  of  plans  at  the  time  to  launch  DMSP-20, 
which  would  help  satisfy  its  requirements  in  the  short-term. 
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Objective  1 :  AOA  Team  Engaged  a  Wide  Range  of 
DOD  Stakeholders;  Some  Concerns  Raised  (continued) 

•  The  Navy  had  concerns  that  the  vignettes  used  to  assess 
minimum  acceptable  values  for  ocean  surface  vector  wind  and 
tropical  cyclone  intensity  data  collection  did  not  capture  the  need 
for  “nowcast”  capability.'9  However,  when  provided  additional  time 
to  assess  the  need  further,  the  Navy  was  unable  to  provide 
quantitative  analysis  supporting  this  need. 

•  The  Joint  Chiefs  of  Staff  and  modeling  experts  were  concerned 
with  the  importance  of  using  global  instead  of  regional  models  to 
assess  capabilities.  Specifically,  while  the  AOA  examined  how 
regional  meteorological  models  would  be  affected  if  data  sources 
were  unavailable,  according  to  officials,  it  did  not  assess  the 
impact  on  global  models. 

•  Stakeholders  had  the  opportunity  to  raise  these  concerns  with  the  Air 
Force  and  CAPE.  The  Air  Force  and  CAPE  acknowledged  these 
concerns  during  the  AOA  process,  such  as  in  Study  Advisory  Group 
meetings,  and  to  some  extent  in  the  “Study  Regrets”  and  “Lessons 
Learned”  sections  of  the  AOA  report.10  However,  these  concerns 
generally  were  not  addressed  further  in  the  study  because  of  time 
constraints,  according  to  DOD  officials. 


9Because  ocean  surface  vector  wind  and  tropical  cyclone  intensity  measurements  can  change  dramatically  within  a  few  hours,  frequent  measurements  are  required  to  maintain 
accurate  “nowcasts,”  which  provide  data  on  current  weather  conditions. 

10The  Study  Advisory  Group  for  the  SBEM  AOA  was  chaired  by  CAPE  and  included  representatives  from  the  Undersecretary  of  Defense  for  Acquisition,  Technology  and  Logistics  and 
other  offices  under  the  Office  of  the  Secretary  of  Defense,  the  Joint  Chiefs  of  Staff,  U.S.  Strategic  Command,  the  military  services,  and  the  intelligence  community. 
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Objective  1:  Limited  Engagement  with  NOAA 
Contributed  to  an  Incorrect  Assumption 

•  Ineffective  coordination  with  NOAA  contributed  to  an  incorrect  assumption 
and  affected  which  alternatives  the  AOA  team  fully  assessed  in  Phase  2. 

•  DOD  did  not  have  a  formal  coordination  and  collaboration  mechanism 
that  specified  roles  and  responsibilities  for  DOD  and  NOAA  for  the 
AOA. 

•  The  Air  Force’s  AOA  Handbook  advises  AOA  leads  to  identify 
appropriate  stakeholders  and  how  they  will  be  involved  in  the  study 
and  review  process,  including  vetting  the  results  of  the  final 
analysis.11  The  handbook  also  advises  AOA  leads  to  identify 
potential  areas  of  risk  pertinent  to  the  study,  such  as  a  lack  of 
stakeholder  participation.  NOAA  is  generally  the  signatory  to 
international  agreements  on  space-based  weather  monitoring  data 
sharing  and  represents  DOD’s  interests  with  international  partners 
on  a  case-by-case  basis,  such  as  for  the  Joint  Polar  System.  As 
such,  NOAA  has  a  stake  in  analysis  of,  and  decisions  about,  the 
use  of  data  from  international  satellites. 


^Office  of  Aerospace  Studies,  Air  Force  Materiel  Command  OAS/A9,  Analysis  of  Alternatives  (AOA)  Handbook:  A  Practical  Guide  to  Analyses  of  Alternatives,  (Kirtland  Air  Force 
Base,  N.  Mex:  July  2010). 
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Objective  1 :  Limited  Engagement  with  NOAA 
Contributed  to  an  Incorrect  Assumption  (continued) 

•  NOAA  was  not  involved  in  reviews  of  the  AOA  or  regular 
discussions  with  the  AOA  study  leadership,  according  to 
NOAA  officials,  and  did  not  provide  formal  input  during  the 
AOA  study  period.  Three  NOAA  staff  participated  in  a 
working  group;  however,  one  of  the  participants  noted  that 
he  received  emails  but  did  not  participate  in  meetings  or 
regular  dialogue  throughout  the  AOA. 

•  Without  effectively  coordinating  with  NOAA  during  the  AOA, 
DOD  made  an  assumption  that  current  capabilities  with 
partner  organizations  would  continue.  This  included  the 
expectation  that  a  European  satellite  providing  coverage  over 
the  Indian  Ocean,  Meteosat-7,  would  be  replaced. 
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Objective  1 :  Limited  Engagement  with  NOAA 
Contributed  to  an  Incorrect  Assumption  (continued) 

•  DOD  established  this  assumption — which  was  based  on 
historical  precedent — during  the  requirements  review.  DOD 
carried  this  assumption  forward  into  the  AOA  without  adequately 
assessing  the  risk  that  European  coverage  might  not  continue. 

•  NOAA  officials,  who  work  closely  with  international  partners, 
had  an  understanding  of  the  plans  for  European  satellites  at 
the  time.  While  DOD  made  some  efforts  to  communicate  with 
NOAA  representatives,  such  as  through  a  discussion  with  a 
senior  technical  advisor,  these  efforts  did  not  result  in  DOD 
determining  the  full  risks  of  assuming  that  Meteosat-7  would 
be  replaced. 

•  In  addition,  during  the  time  of  the  AOA  study,  publicly 
available  reports  from  an  international  coordination  group 
indicated  uncertainty^ about  extended  Meteosat  coverage  over 
the  Indian  Ocean. 12The  reports  stated  that  such  coverage 
was  not  guaranteed  and  that  the  European  agency  had  not 
committed  to  providing  coverage  in  the  long  term. 

12The  Coordination  Group  for  Meteorological  Satellites  has  a  range  of  international  member  organizations,  including  NOAA  and  the  European  agency  responsible  for  the  Meteosat 
satellites,  EUMETSAT.  The  group  meets  about  once  a  year  and  coordinates  satellite  systems  to  support  operational  weather  monitoring  and  forecasting.  Coordination  Group  for 
Meteorological  Satellites,  Report  of  the  41st  Meeting  of  the  Coordination  Group  for  Meteorological  Satellites  (Tsukuba,  Japan:  July  8-12,  2013);  EUMETSAT’s  Plans  for  Indian  Ocean 
Coverage  Beyond  2013,  CGMS-41  EUM-WP-15  via  (July  2,  2013);  and  Report  of  the  40th  Meeting  of  the  Coordination  Group  for  Meteorological  Satellites  (Lugano,  Switzerland:  Nov. 
5-8,2012). 
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Objective  1 :  Limited  Engagement  with  NOAA 
Contributed  to  an  Incorrect  Assumption  (continued) 


•  The  AOA  study  guidance  directed  the  study  team  to  conduct  a  comprehensive 
assessment  of  existing  and  planned  future  international  partner  capabilities — 
including  Meteosat  systems — and  the  likelihood  of  availability  of  such  capabilities. 

•  CAPE’s  assessment  of  the  AOA  stated  the  study  team  could  not  adequately 
assess  the  future  state  of  affairs  with  regard  to  civil  and/or  international 
partners  and  therefore  did  not  attempt  to  quantify  the  risk. 

•  Despite  the  study  team’s  inability  to  quantify  the  risk,  as  well  as  a  statement 
by  the  European  agency’s  Director  of  Operations  that  a  decision  to  replace 
Meteosat-7  had  not  been  made  and  would  hinge  on  several  outstanding 
factors  documented  in  the  AOA  report,  the  final  AOA  report  deemed  the 
likelihood  of  the  gap  not  being  filled  as  low,  based  on  historical  trends.  The 
AOA  report  also  noted  that  the  consequence  of  not  filling  the  gap  could  be 
significant. 

•  DOD  officials  stated  that  assumptions  must  be  made  for  the  purpose  of 
conducting  analysis,  and  that  decisions  based  on  the  AOA  results  are  dependent 
on  the  risk  tolerance  of  the  decision  makers. 

•  While  making  assumptions  is  a  necessary  step  in  scoping  an  AOA  study, 
understanding  the  true  risk  of  an  assumption  about  the  reliability  of  partner 
satellites  is  critical  for  those  making  decisions  based  on  the  SBEM  AOA, 
particularly  if  decision  makers  are  asked  to  use  their  risk  tolerance  to  inform  their 
decision  on  a  DOD  investment. 
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Objective  1:  Range  of  Alternatives  Primarily 
Focused  on  a  Subset  of  Capabilities 

•  The  analysis  of  military  utility  in  Phase  1  informed  the  range  of 
alternatives  to  be  assessed  in  Phase  2.  Based  on  the  Phase  1 
analysis,  the  AOA  team  determined  that  all  but  1  of  the  12 
capabilities  demonstrated  military  utility.13  Within  the  remaining 
11  capabilities,  the  team  determined: 

•  8  capabilities  had  limited  space-based  utility,  would  be 
sufficiently  provided  by  climatology  and/or  ground-based 
capabilities,  or  would  be  sufficiently  provided  by  U.S.  civil 
government  and  international  partner  satellites;  and 

•  3  capabilities  could  be  provided  by  a  DOD  space-based 
solution. 

•  With  approval  from  the  Study  Advisory  Group,  the  AOA  team 
identified  a  total  of  ten  alternatives  to  assess  in  Phase  2. 14 

13The  analysts  determined  that  no  space-based  auroral  characterization  data  were  being  used  operationally  and  therefore  excluded  the  capability  from  further  analysis. 

14The  AOA  included  four  principal  alternatives,  with  six  excursions  under  the  second  principal  alternative,  for  a  total  of  ten. 
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Objective  1:  Range  of  Alternatives  Primarily 
Focused  on  a  Subset  of  Capabilities  (continued) 

•  The  team  determined  that  the  three  capabilities  that  potentially 
justified  a  DOD  solution — 1)  ocean  surface  vector  wind,  (2) 
tropical  cyclone  intensity,  and  (3)  energetic  charged  particles — 
would  be  fully  assessed  in  Phase  2.  The  team  assessed  various 
ways  to  provide  these  three  capabilities  in  seven  alternatives.15 

•  Within  the  scope  of  these  three  capabilities,  the  AOA  team 
duly  evaluated  a  broad  range  of  options  in  Phase  2.  For 
example,  the  team  evaluated  disaggregated  options  with 
sensors  on  separate  and/or  host  satellites;  miniaturized 
satellites  called  CubeSats;  providing  no  capabilities  or  a 
subset  of  the  three  capabilities;  ancrsatellites  in  different 
orbits. 

•  In  prior  work,  based  on  an  assessment  of  over  20  programs 
that  conducted  AOAs,  we  found  that  programs  that 
considered  a  broad  range  of  alternatives  tended  to  have 
better  cost  and  schedule  outcomes  than  programs  that 
examined  a  limited  range  of  alternatives.16 


15These  three  capabilities  were  assessed  under  one  principal  alternative,  with  six  excursions,  for  a  total  of  seven  alternatives. 
16GAO-09-665. 
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Objective  1 :  AOA  Team  Assessed  Alternatives 
for  Other  Capabilities  to  a  Limited  Extent 

•  The  AOA  team  determined  the  remaining  eight  capabilities 
with  military  utility  would  undergo  an  abbreviated 
assessment  in  Phase  2. 

•  One  of  the  ten  alternatives,  referred  to  as  the  baseline, 
assessed  ways  to  mitigate  all  capabilities  by  relying  only 
on  non-DOD  assets;  that  is,  U.S.  civil  government, 
commercial,  and  international  partner  assets. 

•  Two  of  the  ten  alternatives  sought  to  address  all 
capabilities — one  by  leveraging  alternative  data  sources 
as  a  more  cost-effective  approach  to  mitigate  potential 
gaps,  and  another  designed  to  meet  all  minimum 
performance  requirements. 
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Objective  1 :  AOA  Team  Assessed  Alternatives  for 
Other  Capabilities  to  a  Limited  Extent  (continued) 

•  Importantly,  the  AOA  did  not  assess  alternatives  for  providing 
cloud  characterization  and  theater  weather  imagery — deemed 
the  highest-priority  capabilities — at  the  level  of  detail  at  which 
it  assessed  alternatives  for  ocean  surface  vector  wind,  tropical 
cyclone  intensity,  and  energetic  charged  particles.  As  noted 
above,  the  analysis  assumed  U.S.  civil  government  and 
international  partner  assets,  such  as  Europe’s  Meteosat, 
would  continue  to  provide  the  necessary  support,  despite 
indications  at  the  time  that  continued  Meteosat  coverage  was 
uncertain. 
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Objective  1:  Range  of  Alternatives  Included  Limited 
Assessment  of  Key  Aspects  of  Space  Systems 

•  Within  the  range  of  alternatives  assessed,  the  analysis  considered  other 
key  aspects  of  space  systems,  including  resiliency  and  ground  segment 
impacts,  to  a  limited  extent. 

•  DOD  has  recently  emphasized  the  importance  of  considering 
resiliency — the  ability  of  a  system  to  support  the  functions  necessary 
for  mission  success  in  spite  of  hostile  action  or  adverse  conditions — 
for  military  space  systems.  The  AOA  included  a  limited  resiliency 
analysis,  but  resiliency  was  not  used  to  compare  alternatives.17 

•  The  AOA  focused  on  the  space  segment  and  did  not  analyze 
alternative  ground  segment  components.  However,  the  study  team 
considered  implications  and  impacts  to  the  ground  segment  and 
determined  these  would  need  to  be  assessed  more  thoroughly  once 
DOD  decides  on  a  solution.  In  prior  work,  we  recommended  DOD 
comprehensively  examine  the  full  range  of  issues  related  to  moving  to 
a  new  approach,  such  as  disaggregation,  which  include  the  impacts  to 
ground  and  other  interconnected  systems.18  At  the  time,  DOD 
concurred  with  our  recommendation. 

17At  the  time  of  the  AOA,  “resiliency”  was  not  yet  defined  across  DOD.  The  analysis  focused  on  the  need  to  protect  DOD  systems  from  adversary  attack  and  the  potential  to  affect  DOD 
capabilities  if  assets  such  as  satellites  were  unavailable,  according  to  officials. 

18GAO,  DOD  Space  Systems:  Additional  Knowledge  Would  Better  Support  Decisions  about  Disaggregating  Large  Satellites,  GAO-15-7  (Washington,  D.C.:  Oct.  30,  2014). 
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Objective  2:  AOA  Informed  Plans  for  Some 
Capabilities,  but  Is  Less  Useful  as  a  Decision 
Making  Tool  for  Two  Critical  Capabilities 
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Objective  2:  AOA  Informed  Some  Solutions  for 
Meeting  Capability  Gaps;  DOD  Is  Assessing  Solutions 
for  Other  Gaps  Outside  of  the  AOA  Process 

•  The  AOA  provided  useful  analysis  of  potential  DOD  space- 
based  solutions  for  3  of  the  11  capabilities  determined  to 
have  military  utility.  DOD  is  proceeding  with  plans  to  provide 
3  capabilities  that  were  fully  assessed  in  the  AOA. 

•  For  the  remaining  8  capabilities,  the  AOA  determined  other 
assets — such  as  ground-based  and  U.S.  civil  government 
and  international  partner  satellites — could  provide  sufficient 
coverage  or  modeling  development  could  help  mitigate  the 
gaps.  Consequently,  the  AOA  is  less  useful  for  informing 
plans  for  2  of  the  highest-priority  capabilities.  DOD  is  now 
re-assessing  solutions  for  these  capabilities  outside  of  the 
AOA  process. 
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Objective  2:  AOA  Provided  Useful  Analysis  for 
Plans  for  Some  Capabilities 

•  Given  an  increasingly  constrained  budget  environment,  with 
the  SBEM  AOA,  DOD  took  a  relatively  new  approach  to 
considering  ways  to  provide  capabilities  in  Phase  2. 

•  Specifically,  DOD  considered  what  was  really  needed 
operationally  and  options  for  leveraging  other  sources  of 
space-based  weather  data  instead  of  p  anning  to  provide 
all  capabilities  on  its  own. 

•  The  AOA  also  identified  non-space  based  solutions  for 
meeting  capabilities,  including  the  development  of 
algorithms  to  more  effectively  process  existing  data. 

•  In  addition,  DOD  conducted  the  analysis  without  identifying 
a  preferred  solution  before  the  AOA,  avoiding  a  flaw  we 
have  found  in  past  AOAs.19 


19ln  prior  work,  we  found  that  some  AOAs  were  limited  because  program  sponsors  had  decided  on  a  preferred  solution  prior  to  the  AOA.  GAO-09-665. 
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Objective  2:  Plans  for  the  Weather  System 
Follow-on  (WSF) 

•  Based  on  the  AOA’s  focus  on  evaluating  solutions  for  three 
capabilities — (1)  ocean  surface  vector  wind,  (2)  tropical 
cyclone  intensity,  and  (3)  energetic  charged  particles — DOD  is 
developing  plans  to  meet  those  needs  with  space-based 
solutions,  though  some  may  not  be  available  in  time  to  avoid 
short-term  gaps,  as  discussed  further  below. 

•  The  Weather  System  Follow-on  (WSF)  is  intended  to  provide 
the  first  two  capabilities,  ocean  surface  vector  wind  and 
tropical  cyclone  intensity,  in  a  morning  polar  orbit.  According  to 
officials,  plans,  which  are  in  development,  include: 
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Objective  2:  Plans  for  WSF  (continued) 


•  Step  1 :  A  technology  demonstration  using  the  Jet 
Propulsion  Laboratory’s  Compact  Ocean  Wind  Vector 
Radiometer,  with  a  planned  launch  in  2017  and  a  3-year 
design  life.  The  demonstration  plans  to  use  a  currently 
available  payload  to  partially  address  the  ocean  surface 
vector  wind  and  tropical  cyclone  intensity  reguirements.20 
The  Air  Force  plans  to  use  Operationally  Responsive 
Space  acquisition  principles,  according  to  officials,  in  order 
to  streamline  decision  making  and  reduce  the  acquisition 
time  line.21 

•  Step  2:  The  WSF  objective  system,  planned  for  launch  in 
2022,  is  envisioned  as  a  single  polar-orbiting  satellite  with 
a  replacement  satellite  every  5-7  years,  providing 
capabilities  until  around  2040. 

“Satellites  carry  one  or  more  payloads,  which  contain  the  technology  to  provide  certain  capabilities,  such  as  a  microwave  sensor  to  collect  data  on  wind  speed. 

21The  Joint  Operationally  Responsive  Space  Office  was  established  in  2007  to  plan  and  prepare  for  the  rapid  deployment  of  space  capabilities  to  meet  urgent  warfighter  needs. 

According  to  the  ORS  Office,  because  it  is  a  technology  demonstration,  the  project  does  not  require  an  acquisition  strategy.  Instead,  the  Air  Force  Program  Executive  Officer  for 

Space  is  the  milestone  decision  authority  and  approved  the  demonstration  in  February  2016. 
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Objective  2:  Plans  for  WSF  (continued) 


•  Because  WindSat,  the  only  SBEM  sensor  that  fully  meets  DOD’s 
ocean  surface  vector  wind  capability  needs,  may  reach  end-of-life  as 
early  as  this  year,  a  gap  in  providing  full  capabilities  is  likely  to  occur 
until  the  objective  system  is  launched. 


The  Space  and  Missile  Systems  Center  is  preparing  an  acquisition 
strategy  for  the  objective  system,  and  expects  to  present  the  strategy 
to  the Tjffice  of  the  Undersecretary  of  Defense  for  Acquisition. 
Technology  and  Logistics  in  early  2016.  However,  the  NDAAfor 
Fiscal  Year  2016  limits  the  availability  of  funds  for  the  WSF  program 
until  DOD  presents  a  plan  to  Congress  for  meeting  cloud 
characterization  and  theater  weather  imagery  requirements  and 
makes  certain  certifications  to  the  congressional  defense 
committees.22 


•  To  address  the  third  capability,  the  Air  Force  plans  to  host  energetic 
charged  particle  sensors  on  all  of  its  satellites  for  programs  that  are  in 
the  pre-system  development  phase.  Planned  dates  of  implementation 
are  expected  to  vary  across  programs,  but  DOD  estimates  to  begin 
implementation  around  2022. 

22Specifically,  not  more  than  50  percent  of  funds  authorized  to  be  appropriated  by  the  NDAA  or  otherwise  made  available  for  fiscal  year  2016  for  research,  development,  test,  and 
evaluation  AF,  for  the  WSF  may  be  obligated  or  expended  until  DOD  provides  a  briefing  on  the  plan  to  the  Congressional  defense  committees,  and  the  Chairman  of  the  Joint  Chiefs  of 
Staff  certifies  to  the  committees  that  the  plan  will  meet  DOD's  requirements  and  will  not  negatively  affect  the  combatant  command  commanders.  Pub.  L.  No.  1 14-92  §1615  (2015). 
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Objective  2:  AOA  Is  Less  Useful  as  a  Decision  Making 
Tool  for  Two  Critical  Capabilities 


•  The  AOA  is  less  useful  for  informing  plans  for  two  critical  needs. 

•  Currently.  DOD  faces  a  near-term  need  for  cloud  characterization 
and  theater  weather  imagery  capabilities  over  the  Indian  Ocean 
given  the  estimated  end  of  life  ot  the  European’s  Meteosat-7  in 
201 7. 23  DOD  did  not  fully  assess  potential  alternative  solutions  to 
meet  these  capability  needs  within  the  AOA  process. 

•  Instead,  outside  of  the  AOA  process,  DOD  is  now  examining 
solutions  to  meet  cloud  characterization  and  theater  weather 
imagery  capabilities  in  the  short  term.  As  shown  in  table  2,  options 
being  considered,  either  alone  or  combined,  that  would  partially 
replace  Meteosat-7  coverage  include: 

•  Continuing  to  rely  on  DMSP  coverage,  along  with  other  polar- 
orbitina  satellites  from  NOAA  and  Europe,  through  the  end  of  life 
for  DMSP-19. 


Using  data  from  a  European  satellite,  Meteosat-8 — moved  over 
Eastern  Europe,  which  would  partially  cover  the  Indian  Ocean 
region — and  begin  using  data  from  Indian  geostationary 
itellites. 


sa 


230fficials  have  noted  that  without  Meteosat-7,  DOD  can  still  meet  the  minimum  requirement  of  a  4-hour  refresh  rate  using  the  combination  of  existing  polar-orbiting  satellites:  DMSP, 
NOAA’s  Joint  Polar  Satellite  System,  and  Europe’s  Meteorological  Operational  satellite.  However,  these  will  not  consistently  provide  the  30-minute  refresh  Meteosat-7  currently  offers. 
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Objective  2:  Short-Term  Space-Based  Options  for 
Cloud  Characterization  and  Theater  Weather  Imagery 


Table  2:  Potential  Short-Term  Options  for  Providing  Cloud  Characterization  and  Theater  Weather 
Imagery  Over  the  Indian  Ocean 


Program/Satellite  (Owner)3 

Period  of  Coverage 

Steps  Required 

Potential  Limitations 

Defense  Meteorological  Satellite 
Program  [DMSP] 

(Department  of  Defense  [DOD]) 

Present  to  2020 

Continue  supporting  DMSP  satellites  in 
orbit,  combined  with  continued  use  of 
U.S.  civil  government  and  European 
polar-orbiting  satellites 

Will  not  meet  refresh  rate  currently  provided  by 
Metoesat-7;  only  provides  partial  geographic 
coverage  of  the  region. 

Meteosat-8 

(European  Organisation  for  the 
Exploitation  of  Meteorological 
Satellites  [EUMETSAT]) 

2016  to  2020 

EUMETSAT  decision  to  move 
Meteosat-8  further  east ,  which  would 
partially  cover  the  Indian  Ocean 

Implement  minimal  changes  to  data 
downlink 

Availability  depends  on  EUMETSAT 
determination  (expected  in  June  2016)  that 
Meteosat-8  is  no  longer  needed  in  its  current 
position;  only  provides  partial  geographic 
coverage  of  the  region. 

INSAT 

(Indian  Space  Research 
Organisation) 

2017  (estimated)  to 
undetermined 
(potential  for  longer- 
term  support)13 

Secure  approval  from  Indian 
government 

Establish  and  maintain  operational 
data  link 

Further  development  of  reliable  data  delivery  may 
be  needed;  only  provides  partial  geographic 
coverage  of  the  region. 

Source:  GAO  analysis  of  DOD,  National  Oceanic  and  Atmospheric  Administration,  and  international  organization  information.  |  GAO-16-252R 


aDOD  has  also  inquired  with  NOAA  about  moving  a  Geostationary  Operational  Environmental  Satellite  over  the  Indian  Ocean  to  provide  support  for  geostationary  coverage.  While 
technically  feasible,  potential  limitations  would  need  to  be  addressed,  including  ensuring  availability  of  a  satellite  as  a  spare  and  funding  to  cover  costs  associated  with  operations 
and  construction  of  a  ground  system. 

bINSAT-3D  was  launched  in  2013  with  a  7-year  mission  life.  The  Indian  Space  Research  Organisation  has  plans  to  build  and  launch  additional  INSAT  satellites. 


Page  41 

GAO-1 6-252R  DOD  Weather  Satellite  Alternatives 


Page  51 


GAO _ 

Objective  2:  Short-Term  Space-Based  Options  for 
Cloud  Characterization  and  Theater  Weather  Imagery 

(continued) 

•  The  options  noted  above  in  table  2  are  short-term  solutions. 

•  For  example,  continuing  to  rely  on  the  DMSP  satellites  already 
in  orbit  is  to  provide  coverage  until  2020,  based  on  the 
estimated  lifespan  of  the  final  satellite  in  orbit,  DMSP-19. 

•  At  the  time  of  the  AOA,  DOD  expected  to  launch  DMSP-20, 
which  would  have  extended  coverage  until  2023-2027, 
depending  on  when  the  satellite  was  launched.  DOD  and 
intelligence  community  officials  we  interviewed  emphasized 
the  importance  of  launching  DMSP-20  to  provide  continued 
coverage  for  military  and  intelligence  community  needs. 

•  However,  due  to  lack  of  funding  for  DMSP-20,  the  Air  Force 
moved  to  terminate  activities  for  the  satellite  in  December 
201 5. 24  The  termination  cannot  be  reversed  because  of 
payload  contamination  once  maintenance  activities  for 
DMSP-20  are  shut  down,  according  to  officials. 

24The  Explanatory  Statement  of  the  Consolidated  Appropriations  Act,  2016,  Pub.  L.  No.  114-113  (2015)  provided  for  a  $89,351,000  reduction  of  the  DMSP  fiscal  year  2016  budget  request 
and  a  rescission  of  $50,000,000  from  the  fiscal  year  2015  procurement  funds  for  DMSP.  [161  Cong.  Rec.  H9693  at  H9928  (Dec.  17,  2015)]  Prior  to  the  termination,  officials  estimated  the 
costs^ojntegrate^tore^^includingjmaintaining^corto 
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Objective  2:  Longer-Term  Plans  Undetermined 


•  Longer-term  plans  for  providing  cloud  characterization  and 
theater  weather  imagery  over  the  Indian  Ocean  are  not  yet 
determined. 


The  Joint  Requirements  Oversight  Council  directed  the 
Air  Force  to  address  cloud  characterization  and  theater 
weather  imagery  parameters  and  mitigation  strateaies  in 
the  capabilities  development  document  for  WSF.2^The 

A  ■  P™™1  |  ■  g  |  ■■  g  a  |  ■ 


Air  Force 
ie  risk  to 
hese  cou 


s  current  v  examining  options.  Depending  on 
erance  of  decision  makers,  officials  stated 
d  include: 


•  Relying  on  U.S.  civil  government  and  international 
partners; 

•  Hosting  a  sensor  on  a  satellite;  or 

•  Building  a  new  DOD  satellite. 


25According  to  some  officials,  related  efforts  include  the  review  and  re-assessment  of  assumptions  in  the  AOA. 
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Objective  2:  Collaboration  with  NOAA  on 
International  Partner  Capabilities 

•  Limited  mechanisms  for  formal  coordination  and  collaboration 
between  DOD  and  NOAA  increase  the  possibility  that,  going 
forward,  DOD  assesses  potential  solutions  involving  international 
satellite  capabilities  without  fully  understanding  the  risks. 

•  Changing  circumstances  in  a  partner  country  may  lead  to 
changes  in  related  plans.  For  example,  since  the  AOAwas 
completed,  according  to  NOAA,  changes  in  program 
priorities  and  budget  constraints  have  led  Japan  to 
reconsider  its  decision  to  develop  a  follow-on  for  GCOM- 
W,  a  satellite  that  supports  DOD  s  snow  depth,  soil 
moisture,  and  tropical  cyclone  intensity  capabilities. 

•  Effective  coordination  and  collaboration  between  DOD 
and  NOAA  would  help  to  ensure  DOD  is  informed  about 
such  changes  as  it  plans  for  ways  to  provide  capabilities 
in  the  future. 
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Objective  2:  Collaboration  with  NOAA  on 
International  Partner  Capabilities  (continued) 

•  NOAA  and  DOD  officials  have  indicated  that  formal  coordination 
and  collaboration  are  key  to  ensuring  DOD’s  interests  are 
represented  with  international  partners.  In  addition,  in  prior  work 
we  concluded  that  unclear  agency  roles  and  responsibilities  can 
pose  a  challenge  to  interagency  collaboration.26 

•  Since  the  conclusion  of  the  AOA  study  period  in  the  fall  of  2013, 
DOD  and  NOAA  officials  state  they  have  increased  their 
communication  by  discussing  ways  to  leverage  data  from 
international  partner  satellites  in  the  near  term.  DOD  and  NOAA 
have  also  discussed  establishing  more  formal  mechanisms  of 
coordination  and  collaboration,  such  as  a  memorandum  of 
understanding  (MOU). 

•  Our  body  of  work  on  interagency  collaboration  has  shown 
that  establishing  formal  coiraborative  mechanisms  and 
documenting  agreements  can  help  provide  clarity  about  roles 
and  responsibilities  and  support  mutual  accountability.27 

26GAO,  Interagency  Collaboration:  Implications  of  a  Common  Alignment  of  World  Regions  among  Select  Federal  Agencies,  GAO-1 1-776R  (Washington,  D.C.:  July  1 1 , 201 1 ). 

27GAO,  Managing  for  Results:  Key  Considerations  for  Implementing  Interagency  Collaborative  Mechanisms,  GAO-12-1022  (Washington,  D.C.:  Sept.  27,  2012). 
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Objective  2:  Collaboration  with  NOAA  on 
International  Partner  Capabilities  (continued) 

•  One  potential  vehicle  for  formalizing  coordination  and 
collaboration  between  DOD  and  NOAA  is  the  recently  re¬ 
established  Committee  for  Operational  Environmental 
Satellites,  led  by  NOAA’s  Office  of  the  Federal  Coordinator  for 
Meteorology. 

•  The  committee  is  intended  to  help  achieve  interagency 
coordination  in  the  planning  for  and  use  of  operational 
environmental  satellites.  DOD  representatives  have 
participated  in  quarterly  meetings,  according  to  officials. 

•  While  the  committee  does  not  necessarily  focus  on 
international  partner  issues,  one  proposed  objective  of  the 
committee  is  to  establish  a  dialogue  with  other  groups, 
including  international  organizations.  Further,  DOD 
officials  have  stated  that  the  committee  is  one  way  DOD 
can  connect  with  NOAA’s  international  affairs  officials. 
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Scope  and  Methodology 


We  obtained  information  from  DOD  officials  at: 

•  Office  of  the  Secretary  of  Defense: 

•  Cost  Assessment  and  Program  Evaluation, 
Washington,  D.C. 

•  Office  of  the  Undersecretary  of  Defense: 

•  Acquisition,  Technology  and  Logistics, 
Washington,  D.C. 

•  Intelligence,  Washington,  D.C. 

•  Policy,  Washington,  D.C. 

•  Joint  Chiefs  of  Staff: 

•  Directorate  of  Force  Structure,  Resource,  and 
Assessment,  Washington,  D.C. 

•  Directorate  of  Intelligence,  Washington,  D.C. 

•  Army: 

•  Deputy  Chief  of  Staff,  Washington,  D.C. 

•  Navy: 

•  Chief  of  Naval  Operations,  Washington,  D.C. 

•  Naval  Research  Laboratory,  Washington,  D.C. 


Air  Force: 

•  Office  of  the  Assistant  Secretary  of  the  Air 
Force,  Acquisition,  Washington,  D.C. 

•  Office  of  the  Assistant  Secretary  of  the  Air 
Force,  Space  /  DOD  Executive  Agent  for  Space 
Staff,  Washington,  D.C. 

•  Air  Staff,  Operations,  Plans  and  Requirements, 
Washington,  D.C. 

•  Air  Staff,  Strategic  Plans  and  Programs, 
Washington,  D.C. 

•  Air  Combat  Command,  557th  Weather  Wing, 
Offutt  Air  Force  Base,  Neb. 

•  Air  Force  Space  Command,  Peterson  Air  Force 
Base,  Colo. 

•  Space  and  Missile  Systems  Center,  Remote 
Sensing  Systems  Directorate,  Los  Angeles  Air 
Force  Base,  Calif. 

•  Operationally  Responsive  Space  Office, 

Kirtland  Air  Force  Base,  N.  Mex. 
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Scope  and  Methodology  (continued) 


We  obtained  information  from  joint  intelligence 
community  officials  at: 

•  National  Geospatial-Intelligence  Agency, 
Springfield,  Va. 

•  National  Reconnaissance  Office,  Chantilly,  Va. 

We  obtained  information  from  U.S.  civil 
government  officials  at: 

•  National  Aeronautics  and  Space 
Administration,  Washington,  D.C. 

•  National  Oceanic  and  Atmospheric 
Administration: 

•  Office  of  the  Federal  Coordinator  for 
Meteorology,  Silver  Spring,  Md. 

•  National  Environmental  Satellite,  Data, 
and  Information  Service,  Silver  Spring, 
Md. 

•  National  Weather  Service,  Silver  Spring, 
Md. 


We  obtained  information  from  industry  officials 
at: 

•  Atmospheric  and  Space  Technology 
Research  Associates  (ASTRA) 

•  Ball  Aerospace  and  Technologies  Corp. 

•  Exelis  Inc.  (as  acquired  by  Harris 
Corporation) 

•  Lockheed  Martin  Space  Systems 

•  Orbital  ATK 
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Scope  and  Methodology  (continued) 

•  To  determine  the  extent  to  which  the  SBEM  AOA  addressed 
input  from  stakeholders  and  assessed  the  range  of 
alternatives  for  potential  solutions: 

•  We  reviewed  relevant  DOD  and  GAO  documents  to 
develop  an  understanding  of  the  requirements  and 
guidance  for  conducting  an  AOA.  Specifically,  we  reviewed 
fne  DOD  Instruction  5000.02,  the  Office  of  Aerospace 
Studies  Analysis  of  Alternatives  Handbook,  the  SBEM  AOA 
guidance  ana  study  plan,  and  prior  GAO  work  on 
conducting  early  acquisition  activities  and  government 
leadership  and  communication  for  cross-cutting  issues  to 
identify  assessment  criteria.28 

•  We  reviewed  the  AOA  documents  and  interviewed  officials 
involved  in  conducting  and  reviewing  the  AOA  to 
understand  how  the  AOA  was  developed. 

28DOD  Instruction  5000.02,  "Operation  of  the  Defense  Acquisition  System"  (Jan.  7,  2015). 

Page  49 

GAO-1 6-252R  DOD  Weather  Satellite  Alternatives 


Page  59 


GAO 


Scope  and  Methodology  (continued) 


•  We  interviewed  users  and  providers  of  DOD  SBEM  data 

(stakeholders) — officials  from  the  military  services,  the  intelligence 
community,  and  U.S.  civil  agencies — to  gain  their  perspectives  on 
how  stakeholder  views  were  incorporated  into  the  AOA.  We  also 
interviewed  industry  officials  to  obtain  their  perspectives  on  ways  to 
effectively  assess  options  for  providing  SBEM  capabilities, 
including  stakeholders  that  should  be  involved. 


•  To  determine  the  extent  to  which  the  AOA  informed  plans  for  providing 
SBEM  capabilities: 


We  reviewed  documents  and  interviewed  DOD  officials  about  plans 
and  decision  making  processes  for  providing  future  SBEM 
capabilities  in  order  to  understand  decisions  made  following  the 


•  We  reviewed  documents  and  interviewed  NOAA  officials  about 
activities  of  the  international  SBEM  community  (including  partners) 
to  understand  the  potential  effects  on  DOD  plans. 
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Scope  and  Methodology  (continued) 

•  We  interviewed  DOD  officials  and  other  stakeholders  about 
potential  alternatives,  if  any,  to  launching  DMSP-20  in  order  to 
provide  coverage  over  the  expected  Indian  Ocean  gap. 

•  Because  DOD  is  still  assessing  options,  data  on  the  most 
cost-effective  approach  are  not  yet  available.  Instead,  we 
obtained  information  about  the  viability  of  implementing 
Dotential  options  and  the  extent  to  which  they  are  expected 
:o  provide  the  necessary  coverage,  and  related  rough 
order  of  magnitude  cost  estimates,  as  available. 
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Enclosure  II:  Comments  from  the  Department  of  Defense 


OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 

3600  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3600 

ACQUISITION  FEB  0  9  2016 


Ms.  Cristina  Chaplain 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington,  DC  20548 

Dear  Ms.  Chaplain: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  Government  Accountability 
Office  (GAO)  Draft  Report,  GAO-16-252R,  “DEFENSE  WEATHER  SATELLITES:  Usefulness 
of  Analysis  of  Alternatives  is  Limited  Due  to  Time  Constraints  and  Ineffective  Coordination,” 
dated  January  6,  2016  (Code  121280).  Detailed  comments  on  the  report  and  its 
recommendations  are  enclosed. 

Sincerely, 

Darlene  Costello 

Acting  Assistant  Secretary  of 

Defense  for  Acquisition 

Enclosure: 

As  stated 
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GAO  Draft  Report 

GAO-16-252R  (GAO  CODE  121280) 

“DEFENSE  WEATHER  SATELLITES:  USEFULNESS  OF  ANALYSIS  OF 
ALTERNATIVES  IS  LIMITED  DUE  TO  TIME  CONSTRAINTS  AND  INEFFECTIVE 

COORDINATION” 

DEPARTMENT  OF  DEFENSE  RESPONSE  TO  THE  REPORT  AND  COMMENTS 
ON  THE  GAO  RECOMMENDATION 


DOD  COMMENTS  ON  THE  REPORT:  The  Department  of  Defense  does  NOT  CONCUR 
with  the  characterization  that  the  Space  Based  Environmental  Monitoring  (SBEM)  Analysis  of 
Alternatives  (AoA)  is  of  limited  usefulness  due  to  time  constraints  imposed  or  that  is  was 
ineffectively  coordinated.  The  Department  does  not  concur  with  the  assertions  summarized 
below  and  made  in  the  draft  GAO  report: 

a)  Statements  asserting  that  detailed  alternatives  were  developed  for  only  Gaps  3,  8,  and  1 1 . 
The  SBEM  AoA  developed  alternatives  capable  of  meeting  all  12  capability  gaps  and 
examined  the  military  utility  and  assessed  the  operational  risk  of  all  12  gaps. 

b)  Statements  asserting  that  time  constraints  were  imposed  that  prevented  analysis  sufficient 
to  inform  materiel  and  non-material  decisions.  The  Senior  Advisory  Group  (SAG)  that 
guided  the  SBEM  AoA  proceeded  methodically,  moving  forward  to  new  stages  of  the 
AoA  study  only  after  achieving  unanimous  agreement  that  the  preceding  stages  had  been 
completed.  Even  with  this  generous  allowance  of  time,  the  SAG  granted  additional  time 
when  the  Navy  raised  concerns  about  one  requirement’s  minimum  acceptable  value. 

c)  Statements  asserting  that  meteorological  models  were  incomplete.  The  SBEM  AoA  used 
meteorological  models  that  are  currently  used  with  the  DoD  Weather  Centrals  and  by 
civil  meteorological  partners. 

d)  Statements  asserting  that  the  SBEM  AoA  did  not  understand  the  risks  to  international 
partner  SBEM  systems,  particularly  the  European  Meteorological  Satellite  7.  The  SBEM 
AoA  was  fully  informed  of  the  status  and  future  plans  of  all  civil  and  international 
partner  systems.  Furthermore,  the  SBEM  AoA  team  had  standing  members  from  the 
National  Oceanic  and  Atmospheric  Administration  (NOAA)  to  ensure  their  analyses  were 
informed  by  current  data  on  these  systems. 

(A  more  detailed  and  specific  explanation  of  the  disparities  between  the  GAO  Draft  Report  and 
the  SBEM  AoA  Report  will  be  provided  separately.) 

GAO  RECOMMENDATION  FOR  EXECUTIVE  ACTION:  To  ensure  DoD  is  sufficiently 
informed  about  the  availability  and  reliability  of  data  from  U.S.  civil  government  and 
international  partner  satellites  as  it  plans  for  future  SBEM  capabilities  that  rely  on  such  satellites, 
we  recommend  that  the  Secretary  of  Defense  ensure  that  leads  of  future  SBEM  planning  efforts 
establish  formal  mechanisms  of  collaboration  with  the  National  Oceanic  and  Atmospheric 
Administration  that  specify  roles  and  responsibilities  and  assure  accountability  for  both  agencies. 
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DOD  RESPONSE  TO  THE  RECOMMENDATION:  CONCUR.  The  recommendation  is 
consistent  with  current  working-level  engagements  between  the  DoD  and  the  NOAA.  The  DoD 
will  invite  senior  NOAA  representatives  to  participate  in  guiding  future  SBEM  planning  efforts 
and  will  maintain  frequent  contact  with  NOAA  during  the  conduct  of  SBEM  acquisition  efforts 
to  ensure  the  DoD  is  informed  of  the  plans  and  operational  status  of  international  partner  SBEM 
systems. 
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Enclosure  III:  Comments  from  the  Department  of  Commerce 


THE  DEPUTY  SECRETARY  OF  COMMERCE 

Washington,  D.C.  20230 


February  4,  20 1 6 


Ms.  Cristina  T.  Chaplain 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  NW 
Washington,  DC  20548 

Dear  Ms.  Chaplain: 

Thank  you  for  the  opportunity  to  review  and  comment  on  the  Government  Accountability 
Office  (GAO)  draft  report  titled  Defense  Weather  Satellites:  Usefulness  of  Analysis  of 
Alternatives  Is  Limited  Due  to  Time  Constraints  and  Ineffective  Coordination 
(GAO-16-252R).  Enclosed  are  the  National  Oceanic  and  Atmospheric  Administration’s 
programmatic  comments  on  the  draft  report. 

If  you  have  any  questions,  please  contact  me  or  Steve  Haro,  Assistant  Secretary  for 
Legislative  and  Intergovernmental  Affairs,  at  (202)  482-3663. 


$\inrerp1v 


Bruce  H.  Andrews 


Enclosure 
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Department  of  Commerce 

National  Oceanic  and  Atmospheric  Administration’s 
Response  to  the  GAO  Draft  Report  Titled 

Defense  Weather  Satellites:  Usefulness  of  Analysis  of  Alternatives  Is  Limited  Due  to  Time 
Constraints  and  Ineffective  Coordination 
(GAO-16-252R,  January  2016) 

General  Comments 

The  National  Oceanic  and  Atmospheric  Administration  (NOAA)  appreciates  the  opportunity  to 
respond  to  the  Government  Accountability  Office  (GAO)  draft  report.  The  report  on  the  Defense 
Weather  Satellites  Analysis  of  Alternatives  (AOA)  captures  the  key  issues  relating  to  the 
usefulness  of  the  AOA  including  calling  into  question  the  understanding  of  risk  that  was 
accepted.  Further,  the  draft  report  noted  that  NOAA  leadership  was  not  involved  in  the  reviews 
of  the  AOA  or  any  regular  advisory  group  meetings. 

We  believe  some  background  context  on  the  Department  of  Defense  (DOD)  responsibilities  for 
the  early  morning  orbit  should  be  included  in  the  report  and  provide  suggested  language  below. 


(121280) 


Page  66 


GAO-1 6-252R  Weather  Satellite  Alternatives 


This  is  a  work  of  the  U.S.  government  and  is  not  subject  to  copyright  protection  in  the 
United  States.  The  published  product  may  be  reproduced  and  distributed  in  its  entirety 
without  further  permission  from  GAO.  However,  because  this  work  may  contain 
copyrighted  images  or  other  material,  permission  from  the  copyright  holder  may  be 
necessary  if  you  wish  to  reproduce  this  material  separately. 
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The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
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Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512- 
4400,  U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room 
7125,  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 

W 

Please  Print  on  Recycled  Paper. 

